TinyLLM is a lightweight implementation of a decoder model, along with a tokenizer, using PyTorch. This project was inspired by Andrej Karpathy's lecture on language modeling.
TinyLLM aims to provide a simple and accessible implementation of a decoder model for natural language processing tasks. It utilizes PyTorch, a popular deep learning framework, to build and train the model.
To install and set up TinyLLM, follow these steps:
- Clone the repository:
$ git clone https://github.com/anirudhr20/tiny-llm.git
- Install the required dependencies:
$ pip install -r requirements.txt
To use TinyLLM in your own projects, follow these guidelines:
- Import the necessary modules:
$ from llm.model import TinyLLM$ from llm.model import TinyTokenizer
-
Training the model:
cd Scripts/ python train.py -
Model Inference:
cd Scripts/ python inference.py
Contributions to TinyLLM are welcome! If you would like to contribute, please follow these steps:
- Fork the repository: https://github.com/anirudhr20/tiny-llm.git
- Create a new branch for your feature:
$ git checkout -b feature-name - Make your changes and commit them:
$ git commit -m "Add feature-name" - Push your changes to your forked repository:
$ git push origin feature-name - Open a pull request on the main repository.
TinyLLM is distributed under the MIT License. See the LICENSE file for more information.