Meta-Learning Research System M.Sc. Thesis · In Progress¶

FastAPI Celery PostgreSQL Qdrant Redis MLflow Docker QLoRA Qwen3

GitHub

The Problem¶

Researchers starting new AI/ML projects face a recurring challenge: given a new problem, which model architecture, hyperparameters, and training strategy should they use? The existing literature is vast and navigating it manually is slow and error-prone. There's no systematic way to extract and reuse the configurations that worked for similar problems.

The Approach¶

This system automates the process of extracting structured metadata from AI/ML research papers and recommending optimal model configurations for new problems. It combines several techniques into a cohesive pipeline:

Multi-LLM Annotation Pipeline¶

Multiple large language models annotate papers in parallel, extracting structured information about datasets, model architectures, hyperparameters, and reported results. Cross-validation between LLM outputs improves annotation quality.

QLoRA Fine-Tuning¶

The system fine-tunes Qwen3-8B using QLoRA (via Unsloth) on the annotated corpus, creating a specialized model that understands the structure of ML research papers and can extract configurations with high accuracy.

Calibrated Active Learning¶

Rather than annotating every paper equally, the system identifies which papers would provide the most informational value and prioritizes them — reducing annotation costs while maximizing coverage.

Vector Search & Recommendation¶

Extracted configurations are stored in Qdrant as vector embeddings. Given a new problem description, the system performs semantic search to find the most relevant existing configurations and recommends starting points.

Key Features¶

• End-to-end pipeline — from raw PDF papers to actionable model recommendations
• QLoRA fine-tuning with Qwen3-8B/Unsloth for efficient domain adaptation
• Calibrated active learning to minimize annotation effort
• Multi-LLM consensus for robust metadata extraction
• Svelte validation UI for human-in-the-loop verification
• Scalable infrastructure — FastAPI + Celery workers, PostgreSQL for metadata, Qdrant for vectors, Redis for task queuing, MLflow for experiment tracking

Architecture¶

graph LR
    A[PDF Papers] --> B[Multi-LLM Annotator]
    B --> C[Active Learning Selector]
    C --> D[QLoRA Fine-Tuning]
    D --> E[Metadata Extraction]
    E --> F[(PostgreSQL)]
    E --> G[(Qdrant Vectors)]
    H[New Problem] --> G
    G --> I[Configuration Recommendations]

Tech Stack¶