Category: Blog
Your blog category
When building machine learning models, most developers focus on model architectures and hyperparameter tuning.
In today’s AI world, data scientists are not just focused on training and optimizing machine learning models.
This post is divided into three parts; they are: • Why Skip Connections are Needed in Transformers • Implementation of Skip Connections in Transformer Models • Pre-norm vs Post-norm Transformer Architectures Transformer models, like other deep learning models, stack many layers on top of each other.
Retrieval-augmented generation (RAG) has shaken up the world of language models by combining the best of two worlds:
MLOps, or machine learning operations, is all about managing the end-to-end process of building, training, deploying, and maintaining machine learning models.
If you’ve been using large language models like GPT-4 or Claude, you’ve probably wondered how they can write actually usable code, explain complex topics, or even help you debug your morning coffee routine (just kidding!).
This post is divided into three parts; they are: • Interpolation and Extrapolation in Sinusoidal Encodings and RoPE • Interpolation in Learned Encodings • YaRN for Larger Context Window Sinusoidal encodings excel at extrapolation due to their use of continuous functions: $$ begin{aligned} PE(p, 2i) &= sinleft(frac{p}{10000^{2i/d}}right) \ PE(p, 2i+1) &= cosleft(frac{p}{10000^{2i/d}}right) end{aligned} $$ You […]
Machine learning workflows often involve a delicate balance: you want models that perform exceptionally well, but you also need to understand and explain their predictions.
