Large Language Models (LLMs) like GPT-4 enhance software development by aiding code generation and debugging, reducing routine coding tasks. They offer code suggestions, identify errors, and propose fixes, making development more efficient. Integrated into development tools, LLMs can automate documentation, manage complex data structures, and generate test cases, boosting speed and code quality. They also improve team collaboration by summarizing discussions and automating developer support responses.
Because of privacy and security concerns, not all organizations can use public LLMs. Sensitive sectors require private models to keep data secure. While large firms can build proprietary models, smaller organizations face high costs and complexity. Open-source LLMs help here by providing advanced language-model capabilities while supporting regulatory compliance and data protection. This democratizes access to AI and fosters collaboration for model improvement and customization.
In this post we use Meta’s Code Llama, a large language model designed for coding. To streamline deployment, we ‘ll use Ollama, which provides an official Docker container. This setup significantly simplifies getting the LLM operational quickly.
You can find the Docker image on Docker Hub:
https://hub.docker.com/r/ollama/ollama
Pull and run the container with:
docker run -d -v ollama:/root/.ollama -p 11434:11434 --name ollama ollama/ollamaYou can add additional parameters as needed.
If you have an NVIDIA GPU, follow the NVIDIA container toolkit configuration guide and run with:
--gpus=allTo ensure the container does not communicate with the outside world, isolate it from any network:
--network noneNow that the container is running, there are two ways to interact with it.
You can run Code Llama directly in the container shell:
docker exec -it ollama ollama run codellamaExample session:
% docker run -d -v ollama:/root/.ollama -p 11434:11434 --name ollama ollama/ollama
50b40a942366f48d6158c66fd2c24c9f2a489cd26c9fdba0a0f5293840c8045d
% docker exec -it ollama ollama run codellama
>>> How do I remove a docker container?
To remove a Docker container, you can use the `docker rm` command. The general syntax is:
docker rm <container-name or ID>
For example:
docker rm my_container
You can also use the `--force` option to forcefully remove the container without prompting for confirmation:
docker rm --force my_container
To remove a stopped container, use the `-f` or `--force` flag:
docker rm -f my_container
Removing a container does not remove data in volumes associated with it. If you want to preserve data, stop the container and handle the volumes separately.
Alternatively, you can use `docker system prune` to remove stopped containers, unused networks, and untagged images. This may also remove unused volumes; review the prompts before confirming.
>>> Send a message (/? for help)Neat!
To use the model from another location—such as a plugin for your IDE or a central server—you can access Ollama via HTTP.
Example curl request to generate output from Code Llama:
curl http://localhost:11434/api/generate -d '{
"model": "codellama",
"prompt":"Write a shell script to remove a docker container."
}'Complete run (streamed responses):
% curl http://localhost:11434/api/generate -d '{
"model": "codellama",
"prompt":"Write a shell script to remove a docker container."
}'
{"model":"codellama","created_at":"2024-04-10T07:38:44.080588634Z","response":"\n","done":false}
{"model":"codellama","created_at":"2024-04-10T07:38:44.232310384Z","response":"Here","done":false}
{"model":"codellama","created_at":"2024-04-10T07:38:44.385408092Z","response":" is","done":false}
{"model":"codellama","created_at":"2024-04-10T07:38:44.538753134Z","response":" an","done":false}
{"model":"codellama","created_at":"2024-04-10T07:38:44.694064134Z","response":" example","done":false}
...
{"model":"codellama","created_at":"2024-04-10T07:38:49.140863636Z","response":" of","done":false}
{"model":"codellama","created_at":"2024-04-10T07:38:49.29585672Z","response":" the","done":false}
{"model":"codellama","created_at":"2024-04-10T07:38:49.449533595Z","response":" container","done":false}
{"model":"codellama","created_at":"2024-04-10T07:38:49.603687428Z","response":" to","done":false}
{"model":"codellama","created_at":"2024-04-10T07:38:49.759067845Z","response":" be","done":false}
{"model":"codellama","created_at":"2024-04-10T07:38:49.915728095Z","response":" removed","done":false}
...I truncated the output for brevity. The streamed JSON responses can be combined or parsed into a more readable format as needed.
If you have questions, need specific code examples, or want additional assistance, leave a comment or send me a message.