Skip links

Type and hit enter

Training

AI

Digital Teaching

Digital Skills

Menu
Recent Posts
Published in: Artificial Intelligence

Running an LLM Locally on your Laptop or Computer

Author
Published on:

Large language models can support learning, teaching, research and everyday work, but access to the most capable services may require paid subscriptions or involve usage limits. Staff and students may therefore be interested in running models locally as a lower-cost alternative that offers greater control over their data, internet access and choice of model. Local models may also appeal to those concerned about the environmental impact of large-scale cloud computing.  

This post will take you through the steps required to access, download, and choose an appropriate model for your computer. LM Studio is recommended as its visual interface is the simplest starting point for most staff and students. 

What does it mean to run a model locally?

A large language model (LLM) is the type of AI model used to generate and work with text. Some models are referred to as ‘open-weight models’. These are AI models whose learned parameters—or “weights”—are available to download. This allows people to run the model on their own computer using your device’s processor and memory, rather than accessing it only through a company’s online service. 

Hosting software is required, and LM Studio provides user-friendly option for accessing, downloading, and using a variety of LLMs. After the LM Studio software and chosen LLM have been downloaded, core chat and document features can operate without an internet connection. In the standard local setup, prompts and documents are processed on the device itself. You will still need internet access to search for, download or update models and software. 

Why run a model locally?

Whilst there are several benefits to running a model locally on your device, it is important to acknowledge any potential drawbacks. Here are a few considerations: 

Advantages

  • No subscription fees Once downloaded, you can normally use the model without paying per message or each month.
  • Greater privacy Your prompts and documents can remain on your computer rather than being sent to an online service.
  • Works offline After downloading the software and model, you can use it without an internet connection.
  • Environmental benefit Using a small, efficient model locally may require fewer computing resources than sending every task to a large cloud-based model, although the actual impact depends on the model, device and source of electricity.
  • More control You can choose which model to use, adjust its settings and remove it whenever you wish.
  • Fewer usage restrictions You are not usually limited by message allowances or service availability.
  • Useful for experimentation Open-weight models allow students and researchers to explore how different models and settings work.

Potential Drawbacks

  • Models may be less capable Laptop-sized models are generally less powerful than leading cloud-based models.
  • Requires suitable hardware Larger models need considerable memory and processing power. Response times can be slower as a result.
  • Uses storage and electricity Model files can require several gigabytes of space, while running them can drain the battery and make the computer warm.
  • Requires some setup You are responsible for downloading models, selecting suitable settings and installing updates.
  • Usually lacks current information A local model normally cannot search the web unless additional tools are connected.
  • Can still produce errors Local models can generate inaccurate, biased or invented information.
  • Licence conditions vary Open-weight does not necessarily mean unrestricted; some models limit modification, redistribution or commercial use.

Recommended Platforms

Two widely used options make running modals locally accessible without requiring you to build anything yourself. 

PlatformBest forWhat to know
LM Studio (recommended) Beginners and visual learnersA desktop interface for finding, downloading and chatting with models. It can also work with local documents and provide a local API.
OllamaStaff or students who may later use coding tools or integrations Available as an app for macOS and Windows and also widely used through Terminal. It can download models, chat with files and connect to other software.

Getting Started

To get started, download and install LM Studio on your device (available for Windows, Mac and Linux). lmstudio.ai/download  

One installed, select ‘Model Search’ to search for and download large language models. There are a few things to know to help you choose an appropriate model for your device. Read the following information and watch the video below for guidance on selecting the right model for you.

Selecting a suitable Model for your device

Finding out your computers basic specifications will help your device what model to choose. Larger models typically require more processing power and memory (RAM), as well as space on your computer.  

  • Mac: Open the Apple menu and select About This Mac. Note the chip and Memory. 
  • Windows: Open Settings > System > About. Note the processor and Installed RAM. To check graphics memory, open Task Manager > Performance > GPU. 
  • Linux: Open your system's Settings or About page and note the processor, RAM and graphics hardware. 

Typically, the list of models will include the company that developed them, the name of the model, and a number followed by a B. This number indicates ‘billion parameters’. Parameters are the values an AI model learns during training. They help the model recognise patterns, understand language and decide what response to generate. You can think of them as billions of small internal settings that shape how the model responds. For example, a model labelled as 3B means that it has approximately 3 billion parameters. 

Models with more parameters can often handle more complex tasks, but they also require more computer memory and processing power. Parameter count is not a guarantee of quality; a well-trained smaller model may perform better than an older or less suitable larger model. As a simple rule, try a 3B model on an 8 GB laptop, an 8B model on a 16 GB laptop, or a 14B model on a 32 GB laptop.  

This is a size of the model, in Gigabytes (GB). As the model will be downloaded and stored on your computer to run locally, you will need to have enough space to store it. 

You will also need to consider the ‘bit format’ of the model. The bit format describes how precisely each model parameter is stored. Reducing that precision is called quantisation. 

Think of it like compressing an image:

  • Fewer bits → smaller and faster, but slightly less accurate
  • More bits → larger and potentially more accurate, but needs more memory.

For example, for an 8B model (a model with 8 billion parameters):  

Format 

Approx. model size 

Typical use 

16-bit 

16 GB 

Highest precision; powerful hardware 

8-bit 

8 GB 

Good quality but memory-intensive 

4-bit 

4–5 GB 

Best balance for most laptops 

2-bit 

2–3 GB 

Very compact, with more noticeable quality loss 


The actual memory required will be higher because the application also needs space for your conversation and other processing.  

You may encounter names such as Q4_K_M: 

  • Q4 means approximately 4-bit
  • K indicates the quantisation method
  • M means the medium-quality variant of that method.

For normal laptop use, Q4_K_M is usually the safest choice: relatively small, reasonably fast and with limited loss of quality. If you have plenty of memory, Q5_K_M or Q6_K may provide slightly better results. 

Video: Getting Started with LM Studio

Using your model

Once you have installed a model on your computer, you can now start to chat.  

Select ‘New Chat’ – in the chatbox, select ‘pick a model’ to access your loaded models. You can switch between models if you have multiple downloaded. 

You can now ask questions and start to use the model. For advice on writing effective prompts to generate better results when using AI, visit our page Prompt Engineering page. 

Quick Troubleshooting

  • The model will not load: Close other applications, choose a smaller parameter count, select Q4 rather than Q8, or reduce the context length.  
  • Responses are extremely slow: Use a smaller model, keep the laptop connected to power and check that LM Studio is using the recommended GPU settings. Automatic settings are best for beginners.  
  • The laptop becomes hot or the battery drains quickly: This is common during sustained local processing. Use a smaller model, shorten the task, keep ventilation clear and connect the charger.  
  • The answers are poor or do not follow instructions: Check that you selected an Instruct or Chat model rather than a Base model. Start a new conversation and give a clear task, context and required output format.  
  • The model invents facts or references: Treat the response as a draft or learning aid, not evidence. Verify it independently and ask the model to distinguish what it knows from what it is uncertain about.  
  • There is not enough disk space: Remove models you no longer use from My Models. Deleting a model file does not uninstall LM Studio. 

Responsible Use at University

  • Check whether AI is allowed for the task before you begin.
  • Keep your own judgement at the centre. You remain responsible for the work you submit.
  • Verify factual claims, quotations, calculations and references using reliable sources.
  • Do not assume that generated citations are real. Locate and read the original source.
  • Record and declare your use of AI when required by your course, school, funder or publisher.
  • Protect personal, confidential, sensitive and unpublished material, even when processing is local.
  • Check the model's licence if you plan to redistribute it or use outputs commercially.

Takeaway: A Safe First Choice

Begin here: Install LM Studio, match the B size to your RAM, and download Q4_K_M. Test it with a short, low-risk task before adding documents or relying on it for work or study.

Official Guidance and Further Information

You may also like