Git Internals — Ultimate Deep-Dive Guide

A complete engineering guide from beginner concepts to Git-core-engineer-level mental models: object database architecture, .git folder internals, performance engineering, distributed systems thinking, and Staff+/Principal platform engineering.

Table of Contents

1. Big Picture

What Git Actually Is

Git is a content-addressable filesystem with a version control system built on top. At its core, Git is a key-value store where:

• Key = SHA-1 hash (40 hex characters, 20 bytes)
• Value = compressed object (blob, tree, commit, or tag)

Everything else — branches, tags, remotes, staging — is built on top of this simple foundation.

┌──────────────────────────────────────────────────────────────┐
│                    Git Mental Model                            │
├──────────────────────────────────────────────────────────────┤
│                                                                │
│  Layer 4: Porcelain (git add, commit, push, pull, merge)     │
│  Layer 3: Refs (branches, tags, HEAD, remotes)               │
│  Layer 2: Object Graph (DAG of commits, trees, blobs)        │
│  Layer 1: Object Database (content-addressable store)        │
│  Layer 0: Filesystem (.git/objects/)                         │
│                                                                │
└──────────────────────────────────────────────────────────────┘

Why Git Exists

Problem: Software development requires tracking changes across time, across people, and across machines — without corruption, without conflicts, and without a single point of failure.

Previous approaches:

Git’s answer: A distributed, immutable, content-addressed, snapshot-based system that:

• Works entirely offline (local-first)
• Never corrupts (cryptographic integrity)
• Branches in O(1) (pointer, not copy)
• Merges intelligently (three-way merge algorithms)
• Scales to millions of files (packfiles, bitmaps)

Core Mental Models

1. Snapshots, Not Diffs

SVN model (delta-based):
  File A:  v1 → Δ1 → Δ2 → Δ3
  File B:  v1 → Δ1 → Δ2

Git model (snapshot-based):
  Commit 1: [tree → blob_A_v1, blob_B_v1]
  Commit 2: [tree → blob_A_v2, blob_B_v1]  ← B unchanged, same blob reused
  Commit 3: [tree → blob_A_v2, blob_B_v2]

Key insight: Git stores FULL snapshots but deduplicates via content hashing.
If a file doesn't change, Git just points to the same blob.
Packfiles add delta compression LATER as an optimization layer.

2. Immutable Objects

Once created, Git objects NEVER change. A blob with hash abc123 will always contain the same content. This provides:

• Integrity — corruption is immediately detectable
• Cacheability — objects can be cached forever
• Distribution — objects can be safely replicated
• Deduplication — identical content always has same hash

3. Directed Acyclic Graph (DAG)

        A ← B ← C ← D   (main)
                  ↖
                   E ← F  (feature)

Every commit points to its parent(s).
The graph can never have cycles (immutability guarantees this).
Merge commits have multiple parents.
The DAG IS the history.

4. Content-Addressable Storage

hash = SHA-1(header + content)

"hello world" → blob → SHA-1 → 95d09f2b10159347eece71399a7e2e907ea3df4f

Same content ALWAYS produces same hash.
Different content ALWAYS produces different hash (collision-resistant).
This is WHY Git can deduplicate and verify integrity.

Lifecycle: File → History

┌──────────────────────────────────────────────────────────────┐
│  Working Directory    Index (Staging)    Object Database       │
│                                                                │
│  1. Edit file ──────────────────────────────────────────────  │
│  2. git add ─────────► Blob created in .git/objects          │
│                         Index updated with blob hash          │
│  3. git commit ──────► Tree object created (snapshot)        │
│                         Commit object created                 │
│                         (points to tree + parent + metadata)  │
│  4. HEAD updated ────► refs/heads/main updated               │
│                         Reflog entry added                    │
│                                                                │
│  Result: Immutable snapshot in the DAG                        │
└──────────────────────────────────────────────────────────────┘

Git vs Other Systems

Why Git Became Dominant

1. Linux kernel needed it — Linus built it for the world’s largest open-source project
2. GitHub — Social coding platform made Git the default
3. Local-first — Every clone is a full backup
4. Branching is free — Feature branches became standard workflow
5. Speed — Optimized for common operations (status, diff, commit)
6. Ecosystem — CI/CD, code review, tooling all built around Git

Why Git Becomes Slow

Git struggles when:

• Millions of files — Index scanning becomes expensive
• Large binaries — No delta compression benefit, bloats packfiles
• Deep history — Graph traversal for blame/log becomes slow
• Many refs — ref advertisement during fetch becomes expensive
• Huge working trees — stat() syscalls dominate

2. Git Architecture Deep Dive

Object Database

Git’s entire data model consists of four object types:

┌──────────────────────────────────────────────────────────────┐
│                    Git Object Types                            │
├──────────────────────────────────────────────────────────────┤
│                                                                │
│  BLOB                                                         │
│    - Raw file content (no filename, no metadata)             │
│    - Deduplicated by content hash                            │
│    - Compressed with zlib                                     │
│                                                                │
│  TREE                                                         │
│    - Directory listing                                        │
│    - Maps names → blobs/subtrees + mode                      │
│    - Represents a single directory snapshot                   │
│                                                                │
│  COMMIT                                                       │
│    - Points to one tree (root snapshot)                       │
│    - Points to parent commit(s)                              │
│    - Contains author, committer, message, timestamp          │
│    - Forms the DAG                                           │
│                                                                │
│  TAG (annotated)                                              │
│    - Points to a commit                                      │
│    - Contains tagger, message, optional GPG signature        │
│    - Named reference with metadata                           │
│                                                                │
└──────────────────────────────────────────────────────────────┘

Object Relationships

                    refs/heads/main
                         │
                         ▼
                    ┌─────────┐
                    │ commit   │ ← SHA: a1b2c3
                    │ tree: x  │
                    │ parent: y│
                    │ author   │
                    │ message  │
                    └────┬─────┘
                         │ tree
                         ▼
                    ┌─────────┐
                    │ tree     │ ← SHA: d4e5f6
                    │ blob: g  │ "README.md"  100644
                    │ tree: h  │ "src/"       040000
                    └──┬───┬──┘
                       │   │
              ┌────────┘   └────────┐
              ▼                     ▼
         ┌─────────┐          ┌─────────┐
         │ blob    │          │ tree     │ "src/"
         │ content │          │ blob: i  │ "index.ts"
         └─────────┘          └────┬─────┘
                                   │
                                   ▼
                              ┌─────────┐
                              │ blob    │
                              │ content │
                              └─────────┘

Content-Addressable Storage

Every object is stored at a path derived from its hash:

SHA-1: 95d09f2b10159347eece71399a7e2e907ea3df4f

Stored at: .git/objects/95/d09f2b10159347eece71399a7e2e907ea3df4f
                        ^^  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
                     first 2   remaining 38 characters
                     = directory  = filename

Object format:
  <type> <size>\0<content>
  
  Example blob:
  "blob 12\0hello world\n"
  
  Then zlib-compressed and written to disk.

Why Content Hashing Matters

DEDUPLICATION:
  - Two files with identical content → same blob
  - Renaming a file → new tree, same blob
  - Same content across branches → stored once

INTEGRITY:
  - If a single bit flips → hash changes → corruption detected
  - git fsck verifies entire object graph
  - No silent data corruption possible

DISTRIBUTION:
  - Objects are globally unique (hash = identity)
  - Any two repos with same hash = same content
  - Safe to replicate without coordination

PERFORMANCE:
  - Object lookup is O(1) (hash → file path)
  - Existence check is O(1) (file exists?)
  - Network transfer only sends missing objects

Refs System

Refs are simple text files mapping names to commit hashes:

.git/refs/heads/main        → "a1b2c3d4..."  (branch)
.git/refs/heads/feature     → "e5f6g7h8..."  (branch)
.git/refs/tags/v1.0         → "i9j0k1l2..."  (lightweight tag)
.git/refs/remotes/origin/main → "m3n4o5p6..." (remote tracking)
.git/HEAD                    → "ref: refs/heads/main" (symbolic ref)

Why branches are lightweight:

• A branch is a 41-byte file (40 hex chars + newline)
• Creating a branch = writing one file
• Switching branches = updating HEAD + checkout
• No copying, no network calls

HEAD

HEAD tells Git “where am I now?“

Normal state (on a branch):
  .git/HEAD contains: "ref: refs/heads/main"
  → Git follows the indirection: HEAD → main → commit hash

Detached HEAD (directly on a commit):
  .git/HEAD contains: "a1b2c3d4e5f6..."
  → Git is on a specific commit, not a branch
  → New commits won't be tracked by any branch
  → DANGER: commits may become unreachable

3. .git Folder Deep Dive

Complete Structure

.git/
├── HEAD                    ← Current branch pointer
├── config                  ← Repository-level configuration
├── description             ← GitWeb description (rarely used)
├── index                   ← Staging area (binary file)
├── packed-refs             ← Packed reference file (optimization)
├── FETCH_HEAD              ← Last fetch result
├── ORIG_HEAD               ← Previous HEAD (before reset/merge)
├── MERGE_HEAD              ← Commit being merged (during merge)
├── CHERRY_PICK_HEAD        ← Commit being cherry-picked
├── REBASE_HEAD             ← Current commit during rebase
├── hooks/                  ← Git hook scripts
│   ├── pre-commit.sample
│   ├── pre-push.sample
│   └── ...
├── info/
│   ├── exclude             ← Local gitignore (not committed)
│   └── refs                ← Helper file for dumb protocol
├── logs/                   ← Reflog data
│   ├── HEAD                ← HEAD movement history
│   └── refs/
│       └── heads/
│           └── main        ← Branch movement history
├── objects/                ← Object database
│   ├── 95/                 ← Loose objects (first 2 hash chars)
│   │   └── d09f2b...      ← Object file (remaining 38 chars)
│   ├── info/
│   │   └── packs          ← Pack metadata
│   └── pack/              ← Packed objects
│       ├── pack-abc123.idx ← Pack index (binary)
│       └── pack-abc123.pack← Pack data (binary)
├── refs/                   ← Reference pointers
│   ├── heads/             ← Local branches
│   │   └── main           ← Branch → commit hash
│   ├── remotes/           ← Remote tracking branches
│   │   └── origin/
│   │       └── main
│   └── tags/              ← Tags
│       └── v1.0
├── modules/               ← Submodule .git directories
└── worktrees/             ← Linked worktree metadata

.git/objects — Object Database

Loose objects:

• One file per object
• Path: objects/<first-2-chars>/<remaining-38-chars>
• Content: zlib-compressed <type> <size>\0<raw-content>
• Created on every git add (blob) and git commit (tree + commit)

Packed objects (.git/objects/pack/):

• Multiple objects compressed into a single packfile
• Created by git gc, git repack, or during clone/fetch
• Format: sorted by type, then delta-compressed
• Index file (.idx) allows O(1) lookup by hash

Loose object lifecycle:
  git add file.txt
    → Creates .git/objects/ab/cdef... (loose blob)
  
  git gc (or automatic gc)
    → Moves loose objects into packfile
    → Deletes loose object files
    → Creates .idx for fast lookup

Performance implications:

• Loose objects: fast write, slow bulk read (many small files)
• Packed objects: slow write (repack), fast bulk read (sequential I/O)
• Git auto-packs when loose objects exceed threshold (~6700)

.git/index — Staging Area

Binary file containing:

• List of all tracked files with their:
  • Path
  • Blob hash (content version)
  • File mode (permissions)
  • Stat cache (mtime, ctime, size, inode, dev)
  • Flags (assume-unchanged, skip-worktree, etc.)

Index binary format (v2):
  Header: "DIRC" <version> <entry-count>
  
  Entry:
    ctime (32-bit sec + 32-bit nsec)
    mtime (32-bit sec + 32-bit nsec)
    dev, ino, mode, uid, gid, size (each 32-bit)
    SHA-1 hash (20 bytes)
    flags (16-bit: name length, stage, etc.)
    name (variable length, NUL-terminated)
    padding (to 8-byte boundary)
  
  Extensions:
    TREE (cached tree for faster commit)
    REUC (resolve undo)
    EOIE (end of index entry)
    
  Checksum: SHA-1 of entire index

Why the index matters:

• git status compares working tree stat() vs index stat cache
• If stat matches → file unchanged (no content hash needed)
• This makes git status fast even with many files
• git commit creates tree from index, not working tree

Scaling problems:

• 1M files = 1M index entries = ~100MB+ index file
• Every git status must stat() every tracked file
• Solution: sparse index (only tracks checked-out paths)

.git/logs — Reflog

Records every ref movement:

.git/logs/HEAD:
a1b2c3 d4e5f6 Author <email> 1716400000 +0000  commit: Fix bug
d4e5f6 g7h8i9 Author <email> 1716400100 +0000  checkout: moving from main to feature

Format: <old-hash> <new-hash> <author> <timestamp> <timezone>\t<action>: <message>

Recovery implications:

• Reflog keeps references to “deleted” commits for 90 days (default)
• git reflog + git reset --hard <hash> recovers lost work
• Even force-push can be recovered locally via reflog

.git/config — Repository Configuration

[core]
    repositoryformatversion = 0
    filemode = true
    bare = false
    logallrefupdates = true
[remote "origin"]
    url = git@github.com:user/repo.git
    fetch = +refs/heads/*:refs/remotes/origin/*
[branch "main"]
    remote = origin
    merge = refs/heads/main

Special HEAD files

4. Why Git Is Fast Deep Dive

Performance Architecture

┌──────────────────────────────────────────────────────────────┐
│              Git Performance Strategies                        │
├──────────────────────────────────────────────────────────────┤
│                                                                │
│  1. CONTENT DEDUPLICATION                                     │
│     Same content = same hash = stored once                   │
│     Renamed file? Same blob, new tree entry.                 │
│     Unchanged file across commits? Points to same blob.      │
│                                                                │
│  2. STAT CACHE (Index)                                        │
│     Stores file metadata (mtime, size, inode)                │
│     git status: stat() → compare to cache                    │
│     If stat matches → skip content hash → FAST               │
│                                                                │
│  3. PACKFILES                                                 │
│     Delta-compress similar objects                            │
│     Sort by type and size for better deltas                   │
│     Single sequential read vs many random reads              │
│                                                                │
│  4. PACK INDEXES                                              │
│     Binary search on sorted hash list → O(log n)            │
│     Fan-out table for faster narrowing → O(1) first step    │
│     Memory-mapped for OS page cache benefit                  │
│                                                                │
│  5. BITMAP INDEXES                                            │
│     Precomputed reachability bitmaps                         │
│     "Which objects are reachable from commit X?"             │
│     Turns O(n) graph walk into O(1) bitmap lookup            │
│                                                                │
│  6. COMMIT GRAPH                                              │
│     Precomputed commit metadata (parents, generation)        │
│     Accelerates log, merge-base, contains queries            │
│     Avoids unpacking commit objects for traversal            │
│                                                                │
│  7. MULTI-PACK INDEX (MIDX)                                   │
│     Single index across multiple packfiles                   │
│     Avoids searching each pack sequentially                  │
│     Enables geometric repacking                              │
│                                                                │
│  8. FILESYSTEM OPTIMIZATION                                   │
│     Objects split into 256 directories (fan-out)             │
│     Avoids huge directory listings                           │
│     mmap() for packfile access                               │
│     OS page cache friendly                                   │
│                                                                │
└──────────────────────────────────────────────────────────────┘

Packfile Architecture

PACKFILE FORMAT:

Header:
  "PACK" (4 bytes magic)
  Version (4 bytes, usually 2)
  Object count (4 bytes)

Objects (repeated):
  Size + Type (variable-length encoding)
  Content:
    - Undeltified: zlib(raw content)
    - OFS_DELTA: zlib(delta against object at offset)
    - REF_DELTA: zlib(delta against object by hash)

Checksum:
  SHA-1 of entire pack

PACK INDEX (.idx) FORMAT (v2):

Fan-out table: 256 entries (count of objects with hash prefix ≤ N)
Sorted hashes: all object hashes in order
CRC32 table: per-object CRC for verification
Offset table: 4-byte offsets into packfile
Large offset table: 8-byte offsets for >2GB packs
Pack checksum + Index checksum

Delta compression strategy:

• Git finds similar objects (by type, size, path hints)
• Computes binary diff (delta) between them
• Stores base + deltas in chain (max depth configurable)
• Newer objects tend to be bases (faster access to recent versions)

Why packfiles are fast:

• Sequential I/O (one large file vs thousands of small files)
• OS can prefetch/cache efficiently
• Delta chains reduce total storage
• Index enables O(log n) random access

Commit Graph Optimization

.git/objects/info/commit-graph

Contains:
  - OID lookup table (find commit by hash)
  - Commit data table:
    - Tree OID
    - Parent positions (not hashes — avoids unpacking)
    - Generation number (topological distance from root)
    - Commit timestamp
  - Fan-out table
  - Bloom filters (changed-path bloom filters)

Performance gain:
  git log --ancestry-path A..B
    Without commit-graph: unpack each commit object, read parents
    With commit-graph: read precomputed table, follow positions
    
  Result: 10-100x faster for history traversal

Bitmap Indexes

Reachability bitmap:
  For commit X, a bitmap where bit N = "object N is reachable from X"

Example:
  Repository has 1M objects
  Bitmap for commit abc123: [1,0,1,1,0,1,0,0,1,...]
  
  "What objects does client need?"
  = my_bitmap XOR client_bitmap
  = objects to send

Performance gain:
  Without bitmaps: full graph traversal O(objects)
  With bitmaps: bitmap operations O(objects/64) — 64x faster
  
Critical for: git clone, git fetch (server-side)

5. Git Object Model Deep Dive

Blob Internals

# Create a blob manually:
echo "hello world" | git hash-object -w --stdin
# → 95d09f2b10159347eece71399a7e2e907ea3df4f

# What's stored:
# Header: "blob 12\0"  (type + space + size + null byte)
# Content: "hello world\n"
# Full: "blob 12\0hello world\n"
# Hash: SHA-1("blob 12\0hello world\n")
# Storage: zlib_compress("blob 12\0hello world\n")

# Verify:
git cat-file -t 95d09f2b    # → blob
git cat-file -s 95d09f2b    # → 12
git cat-file -p 95d09f2b    # → hello world

Key insight: Blobs have NO filename, NO permissions. Those live in tree objects. This is why renaming a file doesn’t create a new blob.

Tree Internals

# Inspect a tree:
git cat-file -p HEAD^{tree}
# 100644 blob 95d09f2b...   README.md
# 040000 tree a1b2c3d4...   src
# 100755 blob e5f6g7h8...   build.sh

# Tree binary format:
# <mode> <name>\0<20-byte-sha>
# Repeated for each entry, sorted by name

# Modes:
# 100644  regular file
# 100755  executable file
# 120000  symbolic link
# 040000  subdirectory (tree)
# 160000  submodule (commit)

Commit Internals

git cat-file -p HEAD
# tree d8329fc1cc938780ffdd9f94e0d364e0ea74f579
# parent 206fa2bd726fc5e14ee6af01be68e50b1fd5ee46
# author Linus Torvalds <torvalds@linux.org> 1716400000 +0200
# committer Linus Torvalds <torvalds@linux.org> 1716400000 +0200
#
# Initial commit

# Merge commit has multiple parents:
# tree ...
# parent abc123...
# parent def456...
# author ...

Commit hash includes:

• Tree hash (snapshot)
• Parent hash(es) (history)
• Author + timestamp
• Committer + timestamp
• Message

Changing ANY of these → different hash → different commit. This is why git rebase creates new commits (different parents = different hash).

Hashing Pipeline

Object creation:

1. Compute header: "<type> <size>\0"
2. Concatenate: header + raw_content
3. Hash: SHA-1(header + raw_content) → 40-char hex
4. Compress: zlib(header + raw_content)
5. Store: .git/objects/<first-2>/<remaining-38>

SHA-256 transition (ongoing):
  - Git now supports SHA-256 repositories
  - Backward-compatible via hash translation tables
  - Most repos still SHA-1 (practical collision resistance sufficient)

Low-Level Commands

# Manually create a commit (plumbing):
echo "hello" | git hash-object -w --stdin          # Create blob
git update-index --add --cacheinfo 100644,<hash>,file.txt  # Stage
git write-tree                                      # Create tree from index
git commit-tree <tree-hash> -m "message"           # Create commit
git update-ref refs/heads/main <commit-hash>       # Update branch

# This is what `git add` + `git commit` does internally.

6. Git Index & Staging Area Deep Dive

Why Git Has a Staging Area

The index serves multiple purposes:

1. Partial commits — Stage only some changes from working tree
2. Performance — Stat cache avoids rehashing unchanged files
3. Merge resolution — Stores multiple stages during conflicts
4. Atomic commits — Build commit incrementally before finalizing

Index Performance

git status performance path:

For each tracked file:
  1. stat(file) → get mtime, size, inode, dev
  2. Compare to cached stat in index
  3. If ALL stat fields match → file unchanged (skip content check)
  4. If stat differs → hash content → compare to index hash
  5. If hash differs → file is modified

On a 100K file repo:
  - 100K stat() syscalls (fast, ~microseconds each)
  - Rarely needs content hashing (stat cache hit rate > 99%)
  - Total: ~100ms for git status

Merge Conflict Stages

During a merge conflict, the index stores up to 3 versions:

Stage 0: Normal (resolved) entry
Stage 1: Common ancestor version
Stage 2: "Ours" version (current branch)
Stage 3: "Theirs" version (being merged)

git ls-files --stage
# 100644 abc123 1  file.txt   ← ancestor
# 100644 def456 2  file.txt   ← ours
# 100644 ghi789 3  file.txt   ← theirs

After resolution:
# 100644 jkl012 0  file.txt   ← resolved

Sparse Index

For huge monorepos (millions of files):

Traditional index: Entry for EVERY tracked file
  → 5M files × ~70 bytes/entry = ~350MB index
  → Every git status scans entire index
  → Extremely slow

Sparse index: Entry for checked-out files + tree entries for rest
  → 50K checked-out files + directory entries for rest
  → ~5MB index
  → git status only stats checked-out files
  → 100x faster for sparse checkout users

Requires: cone-mode sparse checkout

7. Git Branching & Merging Internals

Branch Mechanics

A branch is a file: .git/refs/heads/<name>
Contents: 40-character commit hash + newline
Size: 41 bytes

Creating a branch:
  echo "a1b2c3d4e5f6..." > .git/refs/heads/new-branch
  
  That's it. No copying, no network, no computation.
  O(1) time, O(1) space.

Switching branches:
  1. Update HEAD to point to new branch
  2. Update working tree to match branch's commit tree
  3. Update index to match new state

Merge Algorithms

Fast-forward merge:

Before:
  A ← B ← C (main)
              ↖
               D ← E (feature)

After (ff):
  A ← B ← C ← D ← E (main, feature)

No merge commit needed — just move the pointer forward.

Three-way merge:

Before:
  A ← B ← C ← F (main)
         ↖
          D ← E (feature)

Merge base: B (common ancestor)
Compare: B→F (main changes) and B→E (feature changes)
Result: new merge commit G with parents F and E

After:
  A ← B ← C ← F ← G (main)
         ↖         ↗
          D ← E ─┘ (feature)

ORT merge strategy (default in Git 2.34+):

• “Ostensibly Recursive’s Twin”
• Replaced recursive strategy
• Better performance on large merges
• Handles rename detection more efficiently
• Cleaner conflict markers

Rebase Internals

Before rebase:
  A ← B ← C ← D (main)
         ↖
          E ← F ← G (feature)

git rebase main (from feature):
  1. Find merge base: B
  2. Compute patches: B→E, E→F, F→G
  3. Reset feature to main (D)
  4. Apply patch B→E → creates E' (new hash, different parent)
  5. Apply patch E→F → creates F'
  6. Apply patch F→G → creates G'

After:
  A ← B ← C ← D (main)
                  ↖
                   E' ← F' ← G' (feature)

Key: E', F', G' are NEW commits (different hashes)
     E, F, G still exist but become unreachable
     Reflog keeps them for 90 days

Why rebase is dangerous in shared branches:

• Rewrites history (new commit hashes)
• Other people’s branches point to OLD hashes
• After force-push, their branches diverge
• Requires coordination to resolve

Cherry-Pick Internals

git cherry-pick <commit>:
  1. Identify commit's parent
  2. Compute diff: parent → commit
  3. Apply diff to current HEAD
  4. Create new commit with same message (different hash)

It's essentially a single-commit rebase.

Rerere (Reuse Recorded Resolution)

git config rerere.enabled true

How it works:
  1. During merge conflict, Git records the conflict state
  2. When you resolve, Git records: conflict_state → resolution
  3. Next time same conflict appears → auto-resolves

Storage: .git/rr-cache/<conflict-hash>/
  - preimage (conflict state)
  - postimage (resolution)

Essential for: long-running branches with repeated merges/rebases

8. Distributed Systems & Networking Deep Dive

Why Git Is Distributed

CENTRALIZED (SVN):
  - Server has truth
  - Client has working copy
  - Operations require network
  - Server down = team blocked

DISTRIBUTED (Git):
  - Every clone has FULL history
  - Every clone can be a server
  - All operations local
  - Network only for sync
  - No single point of failure
  - Offline-first by design

Git Protocol

Smart protocol (standard):

FETCH:
  Client → Server: "I want refs/heads/main"
  Server → Client: "Here's what I have" (ref advertisement)
  Client → Server: "I have X, Y, Z" (have/want negotiation)
  Server → Client: Packfile with missing objects

PUSH:
  Client → Server: "I want to update refs/heads/main to <hash>"
  Client → Server: Packfile with new objects
  Server: Validates, updates refs
  Server → Client: "ok" or "rejected"

Protocol v2 improvements:

• Server-side filtering (only advertise requested refs)
• Reduces overhead for repos with many refs
• Supports partial clone and sparse fetch
• More extensible command structure

Fetch Negotiation

Problem: How to determine which objects client already has?

Algorithm (multi-ack):
  Client: "I want <remote-head>"
  Client: "I have <commit-1>"
  Server: "ACK <commit-1>" (I have that too)
  Client: "I have <commit-2>"
  Server: "NAK" (I don't have that)
  ...continue until common ancestor found...
  Server: sends packfile with objects after common ancestor

Optimization:
  - Commit graph helps estimate common ancestors
  - Bitmap indexes help compute reachable objects
  - Multi-ack reduces round trips

Shallow & Partial Clones

SHALLOW CLONE (--depth N):
  - Only fetches last N commits
  - Creates .git/shallow file listing "graft" points
  - History stops at graft points
  - Use case: CI/CD (only need latest code)
  - Limitation: can't do full blame/log

PARTIAL CLONE (--filter=...):
  - Fetches commit graph but omits some objects
  - Blobs fetched on-demand when accessed
  - Filters:
    --filter=blob:none     (no blobs, fetch on access)
    --filter=blob:limit=1m (no blobs >1MB)
    --filter=tree:0        (no trees, fetch on access)
  - Use case: huge repos where you only need subset
  - Requires server support (protocol v2)

SPARSE CHECKOUT (not a clone type):
  - Full clone, but only checks out subset of files
  - All objects in packfile, just not in working tree
  - .git/info/sparse-checkout defines included paths
  - Cone mode: only include directory patterns (faster)

9. Git Performance & Scaling Engineering

Monorepo Scaling Challenges

┌──────────────────────────────────────────────────────────────┐
│         Why Monorepos Stress Git                              │
├──────────────────────────────────────────────────────────────┤
│                                                                │
│  FILE COUNT (e.g., 5M files):                                │
│    - Index: 5M entries × ~70 bytes = 350MB                   │
│    - git status: 5M stat() syscalls = minutes                │
│    - Solution: sparse checkout + sparse index                │
│                                                                │
│  HISTORY DEPTH (e.g., 1M commits):                           │
│    - git log: traverses DAG                                  │
│    - git blame: needs full history                           │
│    - Solution: commit-graph, shallow clone for CI            │
│                                                                │
│  PACKFILE SIZE (e.g., 50GB):                                 │
│    - Clone: downloads entire pack                            │
│    - gc/repack: needs memory for delta computation           │
│    - Solution: multi-pack index, geometric repack            │
│                                                                │
│  REF COUNT (e.g., 100K branches):                            │
│    - Fetch: ref advertisement sends ALL refs                 │
│    - Solution: protocol v2 (server-side filtering)           │
│    - Solution: packed-refs file                              │
│                                                                │
│  LARGE BINARIES:                                              │
│    - No delta benefit (random-looking data)                   │
│    - Every version stored fully in pack                      │
│    - Solution: Git LFS (pointer files + external storage)    │
│                                                                │
└──────────────────────────────────────────────────────────────┘

Enterprise Scaling Strategies

Git LFS Architecture

WITHOUT LFS:
  Binary file (50MB) stored as blob in packfile
  Every version = another 50MB (no delta benefit)
  Clone downloads ALL versions (500MB for 10 versions)

WITH LFS:
  Pointer file in Git: "oid sha256:abc123...\nsize 52428800\n"
  Actual content stored on LFS server (S3, etc.)
  Clone only downloads pointers (tiny)
  Checkout fetches actual content on demand

Trade-offs:
  + Much smaller repo
  + Clone/fetch is fast
  - Requires LFS server infrastructure
  - Checkout requires network for large files
  - Can't work fully offline with large files
  - Adds operational complexity

CI/CD Clone Optimization

# Fastest CI clone (only need to build):
git clone --depth=1 --single-branch --branch=main <url>
# Downloads: last commit + its tree + needed blobs only

# With partial clone (need some history):
git clone --filter=blob:none --single-branch <url>
# Downloads: all commits + trees, blobs on demand

# For monorepo CI (only need subset):
git clone --filter=blob:none --sparse <url>
git sparse-checkout set packages/my-app
# Downloads: commits + trees, only blobs for my-app

# Performance comparison (hypothetical 10GB repo):
# Full clone:           10GB,  5 minutes
# Shallow depth=1:      200MB, 15 seconds
# Partial + sparse:     50MB,  5 seconds

10. Git Plumbing Commands Deep Dive

Essential Plumbing Commands

# === OBJECT INSPECTION ===

git cat-file -t <hash>          # Object type (blob/tree/commit/tag)
git cat-file -s <hash>          # Object size
git cat-file -p <hash>          # Pretty-print object content
git cat-file --batch            # Batch mode (stdin → stdout)

# === OBJECT CREATION ===

git hash-object -w <file>       # Create blob from file
git hash-object --stdin         # Create blob from stdin (no write)
git mktree                      # Create tree from stdin
git commit-tree <tree> -p <parent> -m "msg"  # Create commit

# === INDEX MANIPULATION ===

git update-index --add --cacheinfo <mode>,<hash>,<path>
git read-tree <tree>            # Load tree into index
git write-tree                  # Create tree from index
git ls-files --stage            # Show index contents

# === REF MANIPULATION ===

git update-ref refs/heads/main <hash>    # Set branch to hash
git symbolic-ref HEAD refs/heads/main    # Set HEAD to branch
git for-each-ref                         # List all refs

# === GRAPH TRAVERSAL ===

git rev-list HEAD               # List all reachable commits
git rev-parse HEAD              # Resolve ref to hash
git merge-base A B              # Find common ancestor

# === MAINTENANCE ===

git fsck                        # Verify object integrity
git gc                          # Garbage collect + repack
git pack-objects                # Create packfile
git unpack-objects              # Extract from packfile
git verify-pack -v <pack>       # Show pack contents
git count-objects -v            # Count loose/packed objects

Manual Commit Workflow

# Create a commit using ONLY plumbing commands:

# 1. Create a blob
echo "Hello, Git internals!" | git hash-object -w --stdin
# → e.g., abc123...

# 2. Stage it in index
git update-index --add --cacheinfo 100644,abc123...,hello.txt

# 3. Create tree from index
TREE=$(git write-tree)
# → e.g., def456...

# 4. Create commit (no parent = initial commit)
COMMIT=$(echo "Initial commit" | git commit-tree $TREE)
# → e.g., ghi789...

# 5. Point branch at commit
git update-ref refs/heads/main $COMMIT

# 6. Set HEAD
git symbolic-ref HEAD refs/heads/main

# Done! Equivalent to: git add hello.txt && git commit -m "Initial commit"

11. Real-World Git Case Studies

GitHub Scaling

Architecture:

• Millions of repositories
• Custom storage layer (DGit → Spokes → now Gitaly-like)
• Repository replication across data centers
• Packfile serving with bitmap indexes
• Server-side commit graphs

Key innovations:

• Bitmap indexes — Invented for GitHub’s scale (makes clone/fetch fast)
• Commit graph — Contributed upstream (accelerates log/blame)
• Protocol v2 — Reduces ref advertisement overhead
• Repository networks — Forks share object storage (deduplication)

Microsoft GVFS / Scalar

Problem: Windows repo (3.5M files, 300GB) couldn’t use Git normally.

Solution (VFS for Git / GVFS):

• Virtual filesystem layer intercepts file access
• Only materializes files on demand
• Prefetch packs for likely-needed objects
• Modified Git to understand partial working trees

Solution (Scalar — lighter approach):

• Sparse checkout + partial clone + commit graph
• Background maintenance (prefetch, gc, commit-graph)
• No kernel driver needed
• Now integrated into core Git

Google Piper

Why NOT Git:

• Single monorepo: billions of files, 80TB+
• Git can’t scale to this (index alone would be enormous)
• Built custom VCS: Piper (centralized, server-side compute)

Lesson: Git has scaling limits. Beyond ~10M files / ~100GB, custom solutions needed.

Facebook Sapling (now open source)

Problem: Mercurial-based monorepo struggling at scale.

Solution:

• Client-side: smart client with lazy fetching
• Server-side: custom backend (Mononoke) for serving
• Virtual filesystem (EdenFS) for working tree
• Compatible with Git hosting

Lesson: At extreme scale, the client must be smart and the filesystem must be virtual.

12. Setup Guide

Git Performance Configuration

# === PERFORMANCE TUNING ===

# Enable commit graph (accelerates log/blame/merge-base)
git config --global core.commitGraph true
git config --global fetch.writeCommitGraph true

# Enable multi-pack index
git config --global core.multiPackIndex true

# Optimize index
git config --global core.untrackedCache true
git config --global core.fsmonitor true  # needs watchman/fsmonitor

# Pack optimization
git config --global pack.threads 0       # use all CPUs
git config --global pack.windowMemory 0  # unlimited (for repack)

# GC optimization
git config --global gc.auto 6700         # auto-gc threshold
git config --global gc.writeCommitGraph true

# === MONOREPO SETUP ===

# Sparse checkout (cone mode)
git sparse-checkout init --cone
git sparse-checkout set packages/my-app shared/

# Partial clone
git clone --filter=blob:none <url>

# Background maintenance (Git 2.29+)
git maintenance start
# Runs: prefetch, commit-graph, loose-objects, incremental-repack

# === CI/CD OPTIMIZATION ===

# Fast clone for CI
git clone --depth=1 --single-branch --branch=$CI_BRANCH $REPO_URL

# With LFS
git lfs install
git clone --filter=blob:none $REPO_URL
git lfs pull --include="path/needed/**"

# === USEFUL ALIASES ===

git config --global alias.lg "log --graph --oneline --all --decorate"
git config --global alias.st "status -sb"
git config --global alias.unstage "reset HEAD --"
git config --global alias.last "log -1 HEAD --stat"
git config --global alias.objects "count-objects -v"

Git LFS Setup

# Install
git lfs install

# Track file types
git lfs track "*.psd"
git lfs track "*.zip"
git lfs track "assets/videos/**"

# Verify
cat .gitattributes
# *.psd filter=lfs diff=lfs merge=lfs -text

# Push/pull works normally
git add file.psd
git commit -m "Add design file"
git push  # LFS uploads to LFS server automatically

Recommended Workflow for Frontend Teams

# For React/Next.js/Astro monorepo:

# 1. Use sparse checkout (only your packages)
git sparse-checkout set \
  packages/web-app \
  packages/design-system \
  packages/shared \
  tools/

# 2. Enable background maintenance
git maintenance start

# 3. Use commit graph
git commit-graph write --reachable

# 4. For CI: minimal clone
git clone --depth=1 --filter=blob:none --sparse $REPO
git sparse-checkout set packages/web-app

# 5. Hooks (via husky/lint-staged)
# Only lint/test changed files, not entire repo
npx lint-staged  # runs on staged files only

13. Tooling Comparison

VCS Comparison

Clone Strategy Comparison

Git Hosting Comparison

14. Cheatsheet

Object Model Quick Reference

┌─────────┬──────────────────────────────────────────────┐
│ Object  │ Contains                                      │
├─────────┼──────────────────────────────────────────────┤
│ blob    │ Raw file content (no name, no mode)          │
│ tree    │ Directory: mode + name + hash (for each entry)│
│ commit  │ tree + parent(s) + author + committer + msg  │
│ tag     │ object + type + tag + tagger + message       │
└─────────┴──────────────────────────────────────────────┘

.git Structure Summary

HEAD           → "ref: refs/heads/main" or commit hash
index          → Binary staging area (stat cache + blob hashes)
objects/       → All Git objects (loose + packed)
refs/heads/    → Branch pointers (name → commit hash)
refs/tags/     → Tag pointers
refs/remotes/  → Remote tracking branches
logs/          → Reflog (history of ref movements)
packed-refs    → Flat file of refs (optimization)
config         → Repo-level settings
hooks/         → Event scripts

Performance Optimization Checklist

☐ Enable commit-graph (fetch.writeCommitGraph=true)
☐ Enable multi-pack-index (core.multiPackIndex=true)
☐ Enable filesystem monitor (core.fsmonitor=true)
☐ Enable untracked cache (core.untrackedCache=true)
☐ Use sparse checkout for monorepos
☐ Use partial clone for CI/CD
☐ Use shallow clone for builds
☐ Run git maintenance start
☐ Use Git LFS for binaries
☐ Configure pack.threads=0 (use all CPUs)

Recovery Checklist

Lost commit after reset:
  git reflog → find hash → git reset --hard <hash>

Deleted branch:
  git reflog → find last commit → git branch <name> <hash>

Corrupted repo:
  git fsck --full → identifies broken objects
  git clone from remote → fresh start

Accidental force push:
  Teammate: git reflog → find old hash → git push -f

Large file committed by mistake:
  git filter-branch or git filter-repo → rewrite history
  (or BFG Repo-Cleaner for simpler cases)

Common Anti-Patterns

❌ Committing node_modules / build outputs
   → Use .gitignore, never commit generated files

❌ Force-pushing to shared branches
   → Only force-push to personal branches

❌ Massive monorepo without sparse checkout
   → Use sparse checkout + partial clone

❌ Storing large binaries without LFS
   → Use Git LFS for anything > 1MB binary

❌ Never running git gc
   → Let auto-gc work, or run maintenance

❌ Rebasing public/shared branches
   → Only rebase local/personal branches

❌ Not using commit-graph on large repos
   → Enable it: git commit-graph write --reachable

15. Real-World Engineering Mindset

Monorepos: Strategy Analysis

Small team (5-15 engineers):

• Full clone works fine
• No special optimization needed
• Focus on good .gitignore and branch hygiene
• Single repo with simple structure

Medium team (15-50 engineers):

• Consider sparse checkout if repo > 1GB
• Enable commit-graph and maintenance
• Use CODEOWNERS for ownership clarity
• Consider Git LFS for design assets
• CI: shallow clone or partial clone

Large team (50-200+ engineers):

• Sparse checkout mandatory
• Partial clone for CI
• Background maintenance scheduled
• Consider Scalar or custom tooling
• Evaluate if monorepo is still right choice
• May need virtual filesystem (EdenFS-like)
• CI: sparse + partial + aggressive caching

Feature Branch Workflows

Strategy: Trunk-based vs Long-lived branches

TRUNK-BASED (recommended for most teams):
  - Short-lived feature branches (1-3 days)
  - Merge to main frequently
  - Feature flags for incomplete work
  - Pros: Less merge conflicts, simpler history
  - Cons: Requires discipline, feature flags complexity

GITFLOW (legacy, avoid for most):
  - develop, release, hotfix branches
  - Complex branching model
  - Pros: Clear release process
  - Cons: Merge hell, slow releases, complex

GITHUB FLOW (simple and effective):
  - main is always deployable
  - Feature branches → PR → merge to main
  - Deploy from main
  - Pros: Simple, fast, low overhead
  - Cons: Needs good CI/CD, feature flags

Large Frontend Repository Strategy

For a React/Next.js/Astro monorepo with design system, multiple apps, shared packages:

Structure:
  packages/
    web-app/          (Next.js)
    marketing/        (Astro)
    design-system/    (React components)
    sdk/              (TypeScript SDK)
    shared/           (utilities)

Git strategy:
  1. CODEOWNERS per package
  2. Sparse checkout: devs clone only their packages
  3. CI: partial clone + sparse checkout per pipeline
  4. Git LFS: design assets (Figma exports, images)
  5. Protected main: require CI pass + review
  6. Trunk-based: short-lived branches, squash merge
  7. Commit-graph + maintenance for performance

16. Brainstorm / Open Questions

Git Architecture (15 questions)

1. Why did Git choose content-addressable storage over filename-addressable?
2. What would break if Git objects were mutable?
3. Why does Git store full snapshots instead of diffs?
4. How does the DAG enable distributed workflows?
5. What are the trade-offs of SHA-1 vs SHA-256 for Git?
6. Why are blobs separated from filenames (stored in trees)?
7. Could Git work without the staging area?
8. Why does Git use zlib compression instead of zstd?
9. What would a “Git 3.0” look like if redesigned today?
10. Why are annotated tags separate objects while branches are just files?
11. How does immutability enable safe garbage collection?
12. What are the implications of Git’s “stupid content tracker” philosophy?
13. Why doesn’t Git track empty directories?
14. How does the object model enable fork networks?
15. What are the theoretical limits of content-addressable storage?

Filesystem Design (15 questions)

16. What filesystem assumptions does Git rely on (POSIX, case-sensitivity)?
17. Why does Git split objects into 256 subdirectories?
18. How do filesystem journaling modes affect Git safety?
19. What happens to Git performance on NFS/network filesystems?
20. Why does the stat cache in the index matter so much?
21. How do inode numbers affect Git’s change detection?
22. What filesystem features does Git NOT use (xattrs, ACLs)?
23. How do filesystem block sizes affect packfile performance?
24. Why does Git struggle on case-insensitive filesystems?
25. How does disk I/O scheduling affect Git operations?
26. What role does the OS page cache play in Git performance?
27. How does SSD vs HDD affect different Git operations?
28. Why does Git use mmap() for packfile access?
29. How do filesystem snapshots (ZFS/Btrfs) interact with Git?
30. What filesystem corruption scenarios can Git detect?

Distributed Systems (15 questions)

31. How does Git achieve eventual consistency without coordination?
32. What’s Git’s CAP theorem position?
33. Why is local-first architecture important for developer tools?
34. How does Git handle the “split-brain” problem?
35. What conflict resolution strategies does Git support?
36. Why doesn’t Git have a built-in consensus mechanism?
37. How do force-pushes violate distributed-system assumptions?
38. What makes Git’s networking protocol efficient?
39. How does object negotiation minimize data transfer?
40. Why can Git be used as a distributed database (bad idea, but why it works)?
41. What are the durability guarantees of a Git repository?
42. How does Git handle clock skew between contributors?
43. What’s the blast radius of a corrupted packfile?
44. How do signed commits add trust to distributed systems?
45. What’s the difference between Git’s distribution and CRDTs?

Performance Engineering (15 questions)

46. Why do packfiles with delta chains improve sequential read performance?
47. What’s the optimal delta chain depth for performance?
48. How do bitmap indexes turn O(n) graph walks into O(1)?
49. Why does commit-graph accelerate merge-base computation?
50. What’s the performance difference between loose and packed objects?
51. How does the index stat cache avoid unnecessary hashing?
52. Why does geometric repacking outperform full repack for append workloads?
53. What makes fsmonitor/watchman important for large repos?
54. How does protocol v2 reduce fetch overhead?
55. Why is git log --follow slow and how could it be improved?
56. What memory pressure does delta compression create during repack?
57. How do multi-pack indexes reduce lookup time?
58. Why is git blame inherently slow and what accelerates it?
59. What’s the cost of maintaining the reflog at scale?
60. How does shallow clone affect subsequent Git operations?

Monorepo Scaling (15 questions)

61. At what scale should you consider alternatives to Git?
62. How does sparse checkout interact with merge conflicts?
63. Why do partial clones need server cooperation?
64. What’s the operational cost of Git LFS at scale?
65. How do virtual filesystems solve the “too many files” problem?
66. What are the trade-offs of polyrepo vs monorepo for Git performance?
67. How does background maintenance affect developer experience?
68. Why do CI pipelines benefit most from clone optimization?
69. What happens when sparse checkout users need to cross boundaries?
70. How do code owners interact with Git’s permission model?
71. What scaling strategy did Microsoft use for the Windows repo?
72. How does Scalar differ from VFS for Git?
73. Why do monorepos need different gc strategies?
74. What’s the impact of many active branches on monorepo performance?
75. How do you measure Git performance degradation over time?

Developer Workflows (15 questions)

76. Why is trunk-based development better for most teams?
77. When is GitFlow actually appropriate?
78. How do feature flags replace long-lived branches?
79. What’s the ideal branch lifetime for merge conflict avoidance?
80. How do signed commits improve supply chain security?
81. Why should CI enforce linear history vs merge commits?
82. What’s the trade-off between squash merge and merge commit?
83. How do pre-commit hooks affect developer velocity?
84. Why is git bisect underused and extremely powerful?
85. How do worktrees improve multi-branch development?
86. What’s the cost of maintaining release branches?
87. How do protected branches interact with automation?
88. Why is commit message convention important at scale?
89. How do monorepo workflows differ from polyrepo workflows?
90. What’s the right level of Git education for a frontend team?

Networking (10 questions)

91. How does pack-objects negotiate which objects to send?
92. Why is the smart protocol more efficient than dumb protocol?
93. What’s the network overhead of ref advertisement?
94. How do shallow boundaries affect fetch performance?
95. Why does protocol v2 support server-side filtering?
96. What’s the bandwidth cost of delta vs full object transfer?
97. How do CDNs interact with Git hosting?
98. What’s the latency impact of multi-round-trip negotiation?
99. How does SSH vs HTTPS affect Git performance?
100. Why do large pushes sometimes timeout?

Object Storage (10 questions)

101. How do alternates objects reduce storage for forks?
102. What’s the space efficiency of packfiles vs loose objects?
103. How does garbage collection identify unreachable objects?
104. What’s the risk of aggressive gc in shared repos?
105. How do replacement objects work for history surgery?
106. What’s the storage cost of signed commits and tags?
107. How does pack-objects choose delta bases?
108. What’s the overhead of maintaining multiple pack files?
109. How do thin packs optimize network transfer?
110. Why does git prune need to be careful about concurrent operations?

Tooling Architecture (10 questions)

111. How does libgit2 differ from shelling out to Git?
112. Why do GUI clients often have different performance than CLI?
113. How do language-specific Git libraries handle the index?
114. What’s the architecture of a Git hosting platform?
115. How do code review tools integrate with Git’s object model?
116. Why is git worktree underused for parallel development?
117. How do IDE Git integrations affect performance?
118. What’s the architecture of a Git proxy/cache?
119. How do Git hooks scale in large organizations?
120. Why do some tools bypass Git and read .git directly?

17. Practice Questions

Beginner (40 questions)

Q1. What does Git store — file diffs or complete snapshots?

• Answer: Complete snapshots (with deduplication via content hashing)
• Why: Each commit points to a tree that represents the full state. Same-content files share blobs.

Q2. True or False: A Git branch is a copy of all files.

• Answer: False
• Why: A branch is a 41-byte file containing a commit hash. Creating a branch is O(1).

Q3. What are the four Git object types?

• Answer: blob, tree, commit, tag
• Why: These four types compose the entire Git data model.

Q4. Where are Git objects stored on disk?

• Answer: .git/objects/ — split into subdirectories by first 2 hex chars of hash.

Q5. What does git add actually do internally?

• Answer: Creates a blob object in .git/objects/ and updates the index (staging area) to point to it.

Q6. What is HEAD?

• Answer: A pointer to the current branch (or directly to a commit in detached state). Stored in .git/HEAD.

Q7. True or False: Deleting a branch deletes the commits on it.

• Answer: False
• Why: Commits still exist in object database. They become unreachable but aren’t deleted until gc runs (after reflog expires).

Q8. What makes Git “distributed”?

• Answer: Every clone contains the full repository history. No server required for most operations.

Q9. What’s the difference between git fetch and git pull?

• Answer: fetch downloads objects + updates remote refs. pull = fetch + merge (or rebase).

Q10. What is the staging area (index)?

• Answer: An intermediate area between working tree and repository. It holds the next commit’s snapshot.

Q11. Why can’t Git track empty directories?

• Answer: Trees only contain entries for blobs (files) and subtrees. No mechanism for empty directory entries.

Q12. What happens when you git clone?

• Answer: Downloads all objects (as packfile), creates remote-tracking branches, checks out default branch.

Q13. True or False: git commit sends data to the server.

• Answer: False. Commits are local. git push sends to server.

Q14. What’s a merge commit?

• Answer: A commit with two or more parents, combining diverged branches.

Q15. What does .gitignore do?

• Answer: Tells Git which untracked files to ignore. Does NOT untrack already-tracked files.

Q16. What is a remote?

• Answer: A named URL pointing to another repository (e.g., origin). Stored in .git/config.

Q17. What’s the difference between --soft, --mixed, and --hard reset?

• Answer: --soft: moves HEAD only. --mixed (default): moves HEAD + resets index. --hard: moves HEAD + resets index + resets working tree.

Q18. What is a fast-forward merge?

• Answer: When current branch is an ancestor of merged branch — just move the pointer forward. No merge commit needed.

Q19. True or False: Git tags and branches are both just pointers to commits.

• Answer: True (lightweight tags are; annotated tags are objects that point to commits).

Q20. What does git stash do internally?

• Answer: Creates commit objects (one for index, one for working tree) on a special ref (refs/stash), then resets working tree.

Q21-Q40: (Additional questions covering: reflog basics, cherry-pick concept, rebase basics, conflict resolution, .gitattributes, blame, bisect, worktrees, submodules, hooks concepts, tag types, remote tracking, upstream, detached HEAD recovery, diff concepts, log filtering, shortlog, archive, bundle, clean)

Junior (40 questions)

Q41. How is a blob’s SHA-1 hash calculated?

• Answer: SHA-1("blob <size>\0<content>") — the hash of the header (type + space + size + null byte) concatenated with raw content.

Q42. What’s the difference between a loose object and a packed object?

• Answer: Loose: individual zlib-compressed file in objects/<2>/<38>. Packed: multiple objects delta-compressed in a single .pack file with .idx for lookup.

Q43. How does git status determine if a file has changed without reading its content?

• Answer: Compares file’s current stat() metadata (mtime, size, inode) against the cached values in the index. Only reads content if stat differs.

Q44. What happens internally during git rebase?

• Answer: Finds merge-base, computes patches for each commit, resets to target, applies patches creating new commits with new hashes (different parents → different identity).

Q45. Explain the three merge conflict stages in the index.

• Answer: Stage 1 = common ancestor, Stage 2 = ours (current branch), Stage 3 = theirs (incoming branch). Stage 0 = resolved.

Q46. How does git gc work?

• Answer: Packs loose objects into packfiles, removes redundant packs, prunes unreachable objects (after reflog expiry), writes commit-graph.

Q47. What is the reflog and how long does it retain entries?

• Answer: Log of all ref movements (branch tip changes, HEAD changes). Default retention: 90 days for reachable, 30 days for unreachable.

Q48. How does git cat-file -p work on different object types?

• Answer: Reads object, decompresses (zlib), strips header, then formats based on type: blob → raw content, tree → mode/name/hash table, commit → structured fields.

Q49. What’s a tree object’s binary format?

• Answer: Repeated entries of: <mode> <name>\0<20-byte-binary-sha>. Sorted by name. No separators between entries.

Q50. Why does renaming a file NOT create a new blob?

• Answer: Blobs store only content (no filename). Filenames are in tree objects. Same content = same blob hash regardless of name.

Q51-Q80: (Additional questions on: packfile delta chains, pack index fan-out table, symbolic refs, detached HEAD dangers, three-way merge algorithm, ORT vs recursive strategy, rerere mechanics, shallow clone limitations, refspec syntax, transfer protocol phases, index extensions, split index, fsmonitor, untracked cache, commit-graph generation numbers, multi-pack index structure, replace objects, grafts vs shallow, alternates, worktree linking)

Senior (40 questions)

Q81. Your monorepo has 2M files. git status takes 30 seconds. Diagnose and fix.

• Answer:
  • Problem: 2M stat() syscalls + large index parse
  • Solutions:
    1. Enable fsmonitor (only stat changed files)
    2. Enable untracked cache (avoid scanning untracked dirs)
    3. Use sparse checkout (reduce tracked files)
    4. Consider sparse index (entries for non-checked-out dirs)
  • Expected improvement: 30s → <1s with fsmonitor + sparse

Q82. Design a CI/CD clone strategy for a 50GB monorepo where builds need only 1 package.

• Answer:

  git clone --filter=blob:none --sparse --single-branch --branch=$BRANCH $URL
  git sparse-checkout set packages/$PACKAGE shared/

  • Downloads: commit graph + trees (~500MB) + blobs for sparse paths (~50MB)
  • Total: ~550MB vs 50GB full clone
  • Time: ~30s vs ~20 min

Q83. Explain how bitmap indexes accelerate git clone.

• Answer: Server precomputes reachability bitmaps for selected commits. During clone, server does bitmap XOR (client_has vs server_has) to determine objects to send. Turns O(objects) graph walk into O(objects/wordsize) bit operations.

Q84. Your team accidentally committed a 2GB video file 6 months ago. How do you remove it from history?

• Answer:
  1. Use git filter-repo --path video.mp4 --invert-paths (rewrites all commits)
  2. Force-push all branches
  3. All team members must re-clone (their refs point to old history)
  4. Verify with git verify-pack -v that file is gone from packs
  5. Prevent recurrence: pre-commit hook checking file size, or Git LFS

Q85. How does geometric repacking differ from full repack?

• Answer:
  • Full repack: combines ALL packs into one, recomputes ALL deltas. O(n²) for delta computation.
  • Geometric repack: maintains packs in geometric size progression (each pack 2x+ previous). Only repacks small packs into next tier. Amortized O(n log n).
  • Better for append-heavy workloads (continuous pushes/fetches).

Q86-Q120: (Additional questions on: pack negotiation optimization, commit-graph bloom filters, multi-pack bitmaps, alternates for fork networks, repository maintenance scheduling, ref backend scalability, SHA-256 migration strategy, server-side hooks for governance, signed push policy, partial clone promisor remotes, sparse index merge conflicts, background prefetch strategy, gc.bigPackThreshold, commitGraph.generationVersion, pack.deltaCacheSize tuning, midx bitmap generation, maintenance task scheduling, repo size monitoring, corruption recovery procedures, cross-DC replication strategies)

Expert / Git Internals Engineer (40 questions)

Q121. Design a Git hosting system that serves 10M repositories with sub-second clone for average repos.

• Answer:
  • Object storage: shared object pools for fork networks (alternates)
  • Caching: precomputed bitmap indexes, commit-graphs for all repos
  • Serving: protocol v2 with server-side ref filtering
  • Replication: 3-way replication across regions
  • Routing: consistent hashing for repo → server mapping
  • Performance: bitmap-accelerated pack generation, want/have negotiation optimized with commit-graph generation numbers
  • Monitoring: per-repo size, pack count, gc age metrics

Q122. What are the trade-offs of delta chains in packfiles? Design an optimal delta strategy.

• Answer:
  • Deep chains: better compression, slower random access (must unpack chain)
  • Shallow chains: worse compression, faster random access
  • Optimal: limit depth to 50 (default), prefer newer objects as bases (recent = accessed more), sort by type then size for better delta candidates
  • For serving: thin packs (only deltas relative to objects client has)
  • Trade-off: CPU at repack time vs storage vs access speed

Q123. How would you design a Git-compatible VCS that scales to 1 billion files?

• Answer:
  • Can’t use traditional index (would be hundreds of GB)
  • Need: virtual filesystem (materialize on access)
  • Need: server-computed trees (client doesn’t build full tree)
  • Need: content-addressed chunks with Merkle DAG (like Git but hierarchical fetching)
  • Need: lazy tree resolution (only expand accessed paths)
  • Examples: Google Piper, Facebook EdenFS, Microsoft GVFS
  • Key insight: at this scale, the working tree must be virtual

Q124-Q160: (Additional expert questions on: custom merge drivers, protocol extensions, pack format v3 design, object storage backends, ref database alternatives (reftable), SHA collision handling, repository federation, cross-repo operations, custom object types, git-annex architecture, CAS optimization, hash function migration, client-server trust model, object verification pipeline, distributed gc coordination, concurrent operation safety, lock-free ref updates, repository archival strategies, object deduplication across repos, pack streaming, delta island optimization, commit signing infrastructure, credential management architecture, hook execution sandboxing, worktree performance characteristics, submodule alternatives, repository observability)

18. Personalized Recommendations

For Your Stack (React, Next.js, Astro, Vite, TypeScript, Monorepos)

Priority Git internals concepts:

1. Monorepo performance — Sparse checkout, partial clone, commit-graph (directly affects your daily workflow)
2. Index mechanics — Understanding why git status is slow with many files (common in monorepos with node_modules issues)
3. Branching/merging — Trunk-based development, squash merge vs merge commit trade-offs
4. CI/CD optimization — Clone strategies, caching, minimal checkout for pipelines
5. Packfiles — Understanding why repos grow and how to manage size

Common mistakes frontend teams make:

1. Committing node_modules or build outputs
2. Not using .gitattributes for line endings (Windows/Mac/Linux team)
3. Large asset files without LFS (design exports, images)
4. Long-lived feature branches creating merge conflicts
5. Not leveraging sparse checkout in monorepos
6. Full clone in CI when shallow/partial would suffice
7. Force-pushing shared branches after rebase

60-Day Learning Plan:

Week 1-2: Object Model Foundations
  - [ ] Manually create blob/tree/commit with plumbing commands
  - [ ] Inspect .git/objects, understand loose vs packed
  - [ ] Run git cat-file on various objects
  - [ ] Create a commit without using git commit

Week 3-4: Index & Performance
  - [ ] Inspect .git/index with git ls-files --stage
  - [ ] Understand stat cache (git status internals)
  - [ ] Enable commit-graph, observe performance difference
  - [ ] Set up fsmonitor (if on large repo)

Week 5-6: Branching & Distribution
  - [ ] Understand merge-base algorithm
  - [ ] Manually trace a three-way merge
  - [ ] Understand rebase = new commits (different hashes)
  - [ ] Trace a fetch operation (object negotiation)

Week 7-8: Scaling & Production
  - [ ] Set up sparse checkout for a monorepo
  - [ ] Configure CI with partial clone + sparse
  - [ ] Set up Git LFS for binary assets
  - [ ] Run git maintenance start, understand tasks
  - [ ] Profile git operations with GIT_TRACE=1

19. Official Documentation & Reference Links

Beginner

• Git Internals - Plumbing and Porcelain
• Pro Git Book (free)
• Git Documentation
• Learn Git Branching (interactive)

Intermediate

• Git Object Model
• Pack Format Documentation
• Index Format
• Git Protocol v2
• Commit Graph Format

Advanced

• Git Source Code
• GitHub Engineering Blog
• GitLab Engineering Blog
• Scalar (Microsoft)
• Git Maintenance Documentation
• Multi-Pack Index
• Bitmap Format

Expert / Git Internals Engineering

• Git Mailing List
• Git Rev News
• Derrick Stolee’s Blog (Git performance)
• Git Merge Conference Talks
• Facebook Sapling
• Monorepo Tools
• VFS for Git (Microsoft)
• Reftable Format
• Git Hash Function Transition

20. Advanced Engineering Topics

Content-Addressable Storage Beyond Git

Git’s CAS model inspired modern infrastructure:

• Docker images — layers addressed by content hash
• Nix store — /nix/store/<hash>-<name> (derivation-addressed)
• IPFS — distributed CAS for files
• Bazel — content-addressed build outputs (remote cache)

The pattern: immutable + content-addressed = cacheable + distributable + verifiable

Future of Version Control

• AI-assisted merging — Using LLMs to resolve semantic conflicts
• Semantic versioning of code — Understanding code meaning, not just text
• Virtual filesystems — Standard (not specialized) approach to huge repos
• Distributed compute — Server-side operations for expensive tasks
• Hash migration — SHA-256 becoming default (post-quantum considerations)
• Cross-repo operations — First-class support for multi-repo workflows
• Reftable — New ref backend replacing files (better performance at scale)

Repository Sustainability

Long-term repository health considerations:

• Growth rate monitoring (size over time)
• Automated maintenance scheduling
• Binary file policies and enforcement
• History rewriting governance
• Archive strategies for inactive repos
• Cost modeling for Git hosting at scale

Summary

Key Takeaways

1. Git is a content-addressable filesystem — Everything is a key-value store with SHA-1 keys
2. Four object types compose all of Git — blob, tree, commit, tag
3. Immutability enables everything — Integrity, deduplication, distribution, caching
4. The DAG IS the history — Commits point to parents, forming a directed acyclic graph
5. Branches are pointers — 41-byte files, O(1) creation, O(1) switching
6. The index is the performance secret — Stat cache avoids content hashing
7. Packfiles are the storage secret — Delta compression + sequential I/O
8. Git scales with optimization layers — commit-graph, bitmaps, MIDX, sparse index
9. Distribution means no single point of failure — Every clone is complete
10. Monorepos need active optimization — Sparse checkout, partial clone, maintenance

Next Steps

1. Explore .git in your own projects with plumbing commands
2. Profile Git operations with GIT_TRACE=1 and GIT_TRACE_PERFORMANCE=1
3. Set up sparse checkout if working in a monorepo
4. Optimize your CI/CD clone strategy
5. Read Git source code for operations you use daily

Advanced Topics to Continue

• Git protocol internals and custom transport
• Pack format v3 and future optimizations
• Reftable backend implementation
• Virtual filesystem integration
• Custom merge drivers and diff drivers
• Git hosting architecture and scaling
• Supply chain security (signed commits, Sigstore)