DEDUPED Hot AI repos, turned into use cases Get 5 a day

What you can do with LLM Architecture From Scratch

A learning repo of PyTorch implementations of modern LLM architectures, with a table comparing each model's normalization, positional encoding, attention type, and use of Mixture of Experts.

Trended earlier

YeonwooSung/llm-architecture-from-scratch on GitHub

Third-party repos we don't own or vouch for. AI-generated use cases can be wrong: review everything and run at your own risk.

What do you want to do?

3 use cases

  • Compare how current LLM architectures are built

    Use the Implemented table to contrast normalization, positional encoding, and attention across models.

    View prompt
    Using the repo YeonwooSung/llm-architecture-from-scratch, compare the rows for <model A> and <model B> in the Implemented table. Output a table of normalization, positional encoding, attention type, and Mixture of Experts use. Verify each cell against the README row before reporting.

  • Trace attention variants from code to table rows

    Map the attention types in the table, such as GQA, sliding window, and MLA, to their implementations.

    View prompt
    In the repo YeonwooSung/llm-architecture-from-scratch, find the folder for <attention type> from the Attention column and summarize how its heads are built. Output a bullet list of code-level differences from Grouped-Query Attention. Verify the folder name matches the README link before reporting.

  • Study Mixture of Experts layers in two models

    Read the MoE implementations in the DeepSeek v3 and Llama 4 Maverick folders to see how experts are used.

    View prompt
    In the repo YeonwooSung/llm-architecture-from-scratch, open the deepseekv3 and llama4_maverick folders and describe how each one implements its Mixture of Experts layer. Output a comparison table of expert structure and routing. Verify each claim against the code in the folder before reporting.

How to use LLM Architecture From Scratch

  1. Check you have what it needs: Python, as listed for the repo; PyTorch, as stated in the repo description; Basic familiarity with transformer attention and normalization layers.
  2. Open YeonwooSung/llm-architecture-from-scratch on GitHub and read the README and license.
  3. Open https://github.com/YeonwooSung/llm-architecture-from-scratch and find the Implemented table.
  4. Note the Normalization, Positional Encoding, Attention, and Mixture of Experts columns for each row.
  5. Pick two rows with different attention types, for example Llama3-8B and DeepSeek v3.
  6. Open the linked folders for both models and read the model code side by side.

These steps come from our top use case, Compare how current LLM architectures are built, which also has a copy-paste agent prompt.

LLM Architecture From Scratch alternatives · Best AI repos for Research

About this repo

Requirements

  • Python, as listed for the repo
  • PyTorch, as stated in the repo description
  • Basic familiarity with transformer attention and normalization layers
Maintain LLM Architecture From Scratch? Add the Featured on Deduped badge

Paste this into your README:

[![Featured on Deduped](https://dedupedai.com/badge/yeonwoosung-llm-architecture-from-scratch.svg)](https://dedupedai.com/use-cases/yeonwoosung-llm-architecture-from-scratch/?ref=badge)

More badge styles and HTML

Questions about LLM Architecture From Scratch

What is LLM Architecture From Scratch?

A learning repo of PyTorch implementations of modern LLM architectures, with a table comparing each model's normalization, positional encoding, attention type, and use of Mixture of Experts. It is an open-source project on GitHub: https://github.com/YeonwooSung/llm-architecture-from-scratch.

Who is LLM Architecture From Scratch for?

Useful for ML engineers and students who want to understand how current open LLM architectures differ at the code level.

What can I do with LLM Architecture From Scratch?

3 use cases so far, for example: Compare how current LLM architectures are built; Trace attention variants from code to table rows; Study Mixture of Experts layers in two models. Each has steps and a copy-paste agent prompt.

What do I need to run LLM Architecture From Scratch?

Python, as listed for the repo; PyTorch, as stated in the repo description; Basic familiarity with transformer attention and normalization layers.

What are alternatives to LLM Architecture From Scratch?

Other hot AI repos in Research: Scientific Agent Skills, Speaker Mem R1, DSH Council, Evo Bot, AI Engineering Course.

Is LLM Architecture From Scratch safe to run?

Deduped does not own, audit or vouch for it. Read the README and license, run it on your own systems first and review every command before you run it.