It has been about three months since the post on the Continue plugin, showing how to run LLM models locally using ollama, so there is a certain amount of information that is out of date. Continue is now much easier to use and there are newer models available to run code generation locally. The latest version of Continue also has inline code completion using the starcoder2 model.
ollama
- Install ollama from https://ollama.com/
- Open a terminal window (these commands were run on an M1 MacBook Pro)
- Run ollama ls This should start ollama and show that no models are available
- Run ollama pull deepseek-coder:6.7b
- Run ollama pull starcoder2:3b
- When this completes run ollama ls again and you should see
NAME ID SIZE MODIFIED
starcoder2:3b f67ae0f64584 1.7 GB 29 seconds ago
deepseek-coder:6.7b ce298d984115 3.8 GB About a minute ago
Continue
- In VSCode, install the Continue plugin from Extensions sidebar (https://continue.dev/)
- Open the Continue sidebar (Command+L can be used if it is not easy to find)
- The models will be loaded automatically. This may take a minute.
- Select Ollama - deepseek-coder:6.7b from the dropdown
- Select the Config gear
- Change allowAnonymousTelemetry to false
Conclusion
Generative AI for code generation is more popular now than three months ago. And protecting digital assets will always be important. Running code generation 100% offline is a safe way to benefit from Generative AI while avoiding the pitfalls of sharing code in the cloud. The Continue plugin, which continues to improve, is a good way to accomplish this.
