\(\rightarrow\) CUDA/C++ learning series:

• GPU architecture and warp scheduling

\(\rightarrow\) LLMs

• Some notes on perplexity and beyond!
• Recent trends to speed up autoregressive inference of LLMs (unifinished).

\(\rightarrow\) Technical

• Handling checkpoints from terminal - some useful tricks.
• Quick reset of my compute pod.
• Speed up your migration to VIM.

\(\rightarrow\) Math:

• Matrix Differential Calculus.
• Convergence Rates of SGD in convex and non-convex cases.

Some GitHub repos:

• GPT2 factorized with (multiple) Kronecker Factors: \(\rightarrow\) Github link.
• Backpropagation from scratch \(\rightarrow\) Github link.
• Randomized Algorithms \(\rightarrow\) Github link.
• Reinforcement Learning \(\rightarrow\) Github link.

Bio/Contact:

📍Passau, Germany.

• I’m a part-time ML research assistant @CAROLL, working on LLMs.
• I’m also a computational math master’s student at @UniPassau.
• Gmail: ayoub.benayad.467 ; LinkedIn ; Instagram.