One CPU
I have solved it with a cpu installation by installing this : https://github.com/krychu/llama
instead of https://github.com/facebookresearch/llama
Complete process to install :
- download the original version of Llama from :
https://github.com/facebookresearch/llama
and extract it to allama-main
folder - download th cpu version from :
https://github.com/krychu/llama
and extract it and replace files in thellama-main
folder - run the
download.sh
script in a terminal, passing the URL provided when prompted to start the download - go to the
llama-main
folder - cretate an Python3 env :
python3 -m venv env
and activate it :source env/bin/activate
- install the cpu version of pytorch :
python3 -m pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cpu #pour la version cpu
- install dependencies off llama :
python3 -m pip install -e .
- run if you have downloaded llama-2-7b :
torchrun --nproc_per_node 1 example_text_completion.py \
--ckpt_dir llama-2-7b/ \
--tokenizer_path tokenizer.model \
--max_seq_len 128 --max_batch_size 1 #(instead of 4)
Comments