Bösherz GoebelAI Patent Hub
de/ EN
Back to workshop overview

Workshop: Local AI

Getting started guide for using local language models.

Article last updated on 3 September 2026

Four pillars for prompt engineering in patent practice

Every task in this course relies on four methods that anchor quality. With agentic tools like OpenCode or Claude Code, Context Engineering runs automatically; in manual prompting you have to maintain it yourself.

Start here: your first local model in 30 minutes

Local AI is the prerequisite for confidential work. The setup is worth it once you want to work on confidential documents (drafting, invention disclosure, internal strategy papers).

What to have ready

  • A machine with at least 8 GB of RAM for the small Gemma 4 variants (E2B, E4B) and Qwen 3 4B, 16 GB for Gemma 4 12B, or 32 GB for Gemma 4 26B
  • Administrator rights to install software
  • A test document (a claim, a short text) for the first conversation

Your path through the course (3 steps)

  1. 10 min install: pick an inference framework from the list below. For beginners I recommend LM Studio or Msty (graphical UI). Download and install.
  2. 10 min model: download a language model. With 8 GB RAM: Gemma 4 E4B. With 16 GB RAM: Gemma 4 12B. With 32 GB RAM: Gemma 4 26B. The first download takes a few minutes.
  3. 10 min first conversation: in the LM Studio / Msty chat, or with Ollama on the command line, send a first prompt: "Explain in plain language what this patent says: [your test document]." Check the result: slow, but fully local.

When you get stuck

  • Hardware not sufficient? Look at the alternatives section: Hugging Face online tests for non-confidential experiments, cloud chatbots for non-confidential documents.
  • Installation stuck? The installation videos below walk through Ollama and LM Studio step by step.
  • What do I need this for in patent practice? The Patent Drafting course builds directly on it.
  • Agent instead of chat? The Agent Framework course explains when OpenCode + Ollama is a step beyond the chat.

Why an inference framework?

Language models like "Llama" or "Gemma" are neural networks stored in special file formats. They cannot be executed directly like other software. An inference framework like Ollama, Msty, or LM Studio handles the work:

  • Model loading: Loads the weight files into memory.
  • Hardware optimization: Adapts the computations to the available hardware (CPU, GPU, specialized AI chips) and uses optimizations such as quantization to reduce memory consumption.
  • API interface: Provides an interface, often compatible with OpenAI standards.
  • Memory management: Swaps parts of the model between RAM and disk when necessary.

Install an inference framework

For beginners I recommend Msty or LM Studio; for advanced users, Ollama.

Ollama

Free

Lightweight, perfect for automation or use with custom apps. Recommended for advanced users.

Download

LM Studio

Free

Comprehensive, with a user-friendly interface and a ChatGPT-like chat interface. Recommended for beginners.

Download

Msty Studio

Core free

Very user-friendly interface, access to the Ollama model library, RAG ("Ask your data"). The free tier covers the core desktop features for individual work; the advanced feature set (Aurum) costs USD 149 per user per year or USD 349 once (as of September 2026).

Download
Caution with confidential data:Make sure to disable any options that send data to the internet. You can monitor and prevent data communication using tools like Microsoft Process Monitor or the Windows Firewall.

Download a language model

The framework does not contain a language model itself. The model has to be downloaded separately. The choice depends on what your hardware can handle and what you intend to use it for.

Gemma 4 (E2B)

Smallest member of the Gemma 4 family, 2.3B effective parameters, 128K context window. Runs on almost any laptop from 8 GB RAM upwards. Good for first tests and simple tasks.

Ollama

Gemma 4 (E4B)

recommended for laptop

Successor to Gemma 3n and the recommendation for laptops. 4.5B effective parameters, 128K context window. 8 GB RAM is enough at Q4 quantization, 16 GB makes it comfortable.

OllamaLM Studio

Gemma 4 (12B)

Dense mid-range model with a 256K context window. An office action plus the prior art fit into a single prompt. Needs 16 GB RAM or 12 GB VRAM.

Ollama

Gemma 4 (26B A4B)

recommended for workstation

Mixture-of-experts model: 26B parameters in total, around 4B active per token. That makes it considerably faster than a dense model of the same size. The context window is 256K. Suitable for local patent drafting. Needs 32 GB RAM or 24 GB VRAM (e.g. RTX 4090). With more memory available, the dense 31B model is the alternative.

Ollama

Qwen 3 (4B)

Compact, Apache 2.0 license, runs on almost any laptop. Useful as a second opinion from a different model family. Qwen generations 3.6 and 3.8 are also on Ollama, but their default tags start at 18 GB and require a workstation.

Ollama
Licensing note:Gemma 4 is released under the Apache License 2.0, whose Section 3 contains an express patent licence from the contributors. The earlier generations Gemma 3 and Gemma 3n remain under the Gemma Terms of Use with use restrictions attached. For deployment in a firm, that is a further reason to move to Gemma 4. The RAM and VRAM figures above refer to the model weights. The context window in use takes additional memory, so filling it needs more than the stated figure. Parameter counts, context windows and licence per Google's Gemma 4 model overview and model card, as of September 2026.

Installation videos

Two short walkthroughs for the setup:

Installation: Ollama

Installation: LM Studio

Alternatives without local installation

Only for experimenting with non-confidential data.The alternatives below run in the cloud. They are useful for trying out models and finding out which one fits your workflow before you invest in local hardware. They are not suitable for client files or invention disclosures.
  • Hugging Face: Repository for open models and datasets, including the ones named here. Many can be tested directly online. Useful to check whether a local installation is worthwhile before buying hardware. To use this option during the seminar, please create a free account beforehand: huggingface.co.
  • Cloud chatbots: For non-confidential data you can also useChatGPT, Microsoft's Copilot (which uses OpenAI models among others), orClaude.

Further resources

Prompt Library

Create, manage and customize your own prompts.

Patent drafting process

Prompts and notes assigned to the individual process steps.

Cheat sheet: GenAI software

Curated cheat sheet of the GenAI software used in the workshop.

Example invention report (coffee mug)

PDF of a sample invention report. Practice material.

Example invention report (poker chip)

More demanding invention report with two prior-art documents (D1, D2), multiple embodiments and reference numerals — useful to stress-test local models on a realistic drafting case.

Bonus

Local image generation with ComfyUI

ComfyUI is a node-based interface for image generation models and runs entirely on your own machine. The same confidentiality logic that applies to local language models applies to images. The core is open source under GPL-3.0, and there is a desktop installer for Windows and macOS. In practice it is useful for illustrations in talks and training material. Worth highlighting for patent work is image-to-image editing, converting a product photo or CAD screenshot into a line drawing. The formal requirements for patent drawings still remain manual work. A concrete workflow recommendation is in the GenAI Tool Finder.

  • Check the licences: the image models are downloaded separately and carry their own licences, ranging from Apache 2.0 to research licences that exclude commercial use. Read the licence of the specific model before using it on client work.
  • Custom nodes are executable code: extensions and third-party workflow files pull in Python that runs with the same rights as ComfyUI itself. Running it in a sandbox is advisable, such as a separate user account with no access to critical data. Install extensions only from known sources and read through third-party workflows before running them.
DownloadDocumentationGitHub

Newsletter

Stay up to date on generative AI in patent practice.

Join the GenAI NewsletterRead all previous issues

Content partially AI-generated, curated by Sebastian Goebel. This is not legal advice but training material for my workshops. No guarantee of accuracy or completeness. Software provided as is.