LLM Frameworks

ZhuLinsen/daily_stock_analysis

LLM 驱动的多市场股票智能分析系统：多源行情、实时新闻、决策看板与自动推送，支持零成本定时运行。 LLM-powered multi-market stock analysis system with multi-source market data, real-time news, decision dashboard, automated notifications, and cost-f

kingcharleslzy-ai/dsh-purge

dsh-purge Community · DeepSeek Harness 社区维护版：宿主兼容、状态诊断、提示词管理、更新与本机备份回滚。基于小杨 / YuJunZhiXue 的 dsh-purge。

career-ops-hq/career-ops

Open-source AI job search agent and job finder: scan job boards, score each job 1-5 against your CV before you apply, tailor an ATS-friendly resume and cover letter, get interview prep and an applicat

ggml-org/llama.cpp

LLM inference in C/C++

vllm-project/vllm

A high-throughput and memory-efficient inference and serving engine for LLMs

QuantumNous/new-api

Open-source AI gateway. Self-hosted LLM gateway with OpenAI-, Claude- and Gemini-compatible APIs, model routing, access control, usage tracking and cost accounting.

rtk-ai/rtk

CLI proxy that reduces LLM token consumption by 60-90% on common dev commands. Single Rust binary, zero dependencies

ccch1mneyyy/dsh-go

dsh-go — DeepSeek Harness 装进口袋：手机本机运行，也能遥控你的电脑。

BerriAI/litellm

The fastest, litest AI Gateway. Rust core with Python SDK. Call 100+ LLM APIs in OpenAI (or native) format with cost tracking, guardrails, load balancing, and logging [Bedrock, Azure, OpenAI, Anthropi

hiyouga/LlamaFactory

Unified Efficient Fine-Tuning of 100+ LLMs & VLMs (ACL 2024)

anywhere-labs/deepseek-harness-desktop

为 DeepSeek Harness (DSH) 插件生态打造的现代化桌面端解决方案。万物皆「插件」，桌面本身也是「插件」。

google/elephant-goldfish-model

Open-source plugin for designing and reviewing software with agents

alibaba/open-code-review

Secure, fast, efficient, battle-tested at Alibaba's scale. Hybrid architecture code review tool: deterministic pipelines + LLM Agent, precise line-level comments, built-in multi-language ruleset (NPE,

Mintplex-Labs/anything-llm

Stop renting your intelligence. Own it with AnythingLLM. Everything you need for a powerful local-first agent experience

Ebony-Vinyl/dsh-our-free-model

在 dsh 里装上这个插件即可，无需登录、注册或填 API Key，就能使用包括 DeepSeek V4.1 Flash、Kimi K3 在内的前沿模型——完全免费，不限量。 All you do is install this plugin in dsh: no login, no sign-up, no API key — the frontier models are just there,

jundot/omlx

LLM inference server with continuous batching & SSD caching for Apple Silicon — managed from the macOS menu bar

stanfordnlp/dspy

DSPy: The framework for programming—not prompting—language models

microsoft/semantic-kernel

Integrate cutting-edge LLM technology quickly and easily into your apps

onyx-dot-app/onyx

Open Source AI Platform - AI Chat with advanced features that works with every LLM

awesome-dsh-plugin/awesome-dsh-plugin

A curated list of plugins for DeepSeek Harness (dsh) · DeepSeek Harness 插件精选列表

browser-use/browser-harness

Let Claude Code, Codex or Cursor use your real Chrome, logged in as you. One prompt to set up. Open source, MIT.

unclecode/crawl4ai

Official Crawl4AI Cloud: a hosted web scraping, extraction, and search API for AI agents. Free API key, 1,000 credits a month, no card. Fully open source: this repo is the Apache-2.0 crawler that powe

microsoft/SkillOpt

SkillOpt is a text-space optimizer that trains reusable natural-language skills for frozen LLM agents through trajectory-driven edits, validation-gated updates, and deployable best_skill.md artifacts.

openai/openai-python

The official Python library for the OpenAI API

google/langextract

A Python library for extracting structured information from unstructured text using LLMs with precise source grounding and interactive visualization.

kvcache-ai/ktransformers

A Flexible Framework for Experiencing Heterogeneous LLM Inference/Fine-tune Optimizations

Fei2411/DS-Bot

A team of long-lived bots for DeepSeek Harness, with main bots, bot-to-bot messages and group chats.

TauricResearch/TradingAgents

TradingAgents: Multi-Agents LLM Financial Trading Framework

OpenBMB/ChatDev

ChatDev 2.0: Dev All through LLM-powered Multi-Agent Collaboration

trailhq/Trail-Code

One shared memory for every coding agent on your team. Claude Code, Cursor and Codex get a map of your code and your team's past decisions, so they pick up where you left off.

mlflow/mlflow

The open source AI engineering platform for agents, LLMs, and ML models. MLflow enables teams of all sizes to debug, evaluate, monitor, and optimize production-quality AI applications while controllin

yamadashy/repomix

📦 Repomix is a powerful tool that packs your entire repository into a single, AI-friendly file. Perfect for when you need to feed your codebase to Large Language Models (LLMs) or other AI tools like C

NVIDIA/TensorRT-LLM

TensorRT LLM provides users with an easy-to-use Python API to define Large Language Models (LLMs) and supports state-of-the-art optimizations to perform inference efficiently on NVIDIA GPUs. TensorRT

LMCache/LMCache

LMCache: Supercharge Your LLM with the Fastest KV Cache Layer

xtekky/gpt4free

The official gpt4free repository | various collection of powerful language models | opus 4.6 gpt 5.3 kimi 2.5 deepseek v3.2 gemini 3

upstash/context7

Context7 Platform -- Up-to-date docs for AI agents

mozilla-ai/llamafile

Distribute and run LLMs with a single file.

MemTensor/MemOS

Self-evolving memory OS for LLM & AI Agents: ultra-persistent memory, hybrid-retrieval, and cross-task skill reuse, with 35.24% token savings and DeepSeek Harness support.

anywhere-labs/dsh-desktop

为 DeepSeek Harness (DSH) 插件生态打造的现代化桌面端解决方案。万物皆「插件」，桌面本身也是「插件」。

dataelement/dsh-desktop

DSHDesktop：DeepSeek Harness Desktop / DeepSeek Harness 桌面版

vllm-project/vllm-omni

A framework for efficient model inference with omni-modality models

mlc-ai/web-llm

High-performance In-browser LLM Inference Engine

modelscope/ms-swift

Use PEFT or Full-parameter to CPT/SFT/DPO/GRPO 600+ LLMs (Qwen3.8, DeepSeek-V4, GLM-5.1, InternLM3, Llama4, ...) and 300+ MLLMs (Qwen3-VL, Qwen3-Omni, InternVL3.5, Ovis2.5, GLM5.3, Gemma4, Llava, Phi4

The-Vibe-Company/quivr

An open-source engine that turns continuous content streams into search and monitoring. Durable ingestion, hybrid search, alerts, and plugins for formats, models and business rules.

langchain4j/langchain4j

LangChain4j is an idiomatic, open-source Java library for building LLM-powered applications on the JVM. It offers a unified API over popular LLM providers and vector stores, and makes implementing too

xming521/WeClone

Build a portrait of your life. Assistants that know you. Clones that talk like you.

datawhalechina/self-llm

《开源大模型食用指南》针对中国宝宝量身打造的基于Linux环境快速微调（全参数/Lora）、部署国内外开源大模型（LLM）/多模态大模型（MLLM）教程

cloudwego/eino

The ultimate LLM/AI application development framework in Go.

deepseek-ai/deepseek-harness

DeepSeek Harness: Everything is a Plugin.

Lightning-AI/litgpt

20+ high-performance LLMs with recipes to pretrain, finetune and deploy at scale.

truefoundry/trueforge

The open-source agent harness - the runtime layer that turns an LLM into a working agent.

rasbt/LLMs-from-scratch

Implement a ChatGPT-like LLM in PyTorch from scratch, step by step

mlc-ai/mlc-llm

Universal LLM Deployment Engine with ML Compilation

deepseek-ai/DeepInfer

DeepInfer: High-performance, large-scale inference framework for DeepSeek models

Orkas-AI/Orkas

Orkas is an open-source (MIT), local-first AI team for your desktop: a Commander turns your goal into to-dos and coordinates 16 built-in specialist agents plus Claude Code, Codex, OpenCode, OpenClaw a

vllm-project/semantic-router

An open, programmable decision layer for models and compute.

simonw/llm

Access large language models from the command-line

blinkospace/blinko

An open-source, self-hosted personal AI note tool prioritizing privacy, built using TypeScript .

nanobrowser/nanobrowser

Open-source AI agent for web automation. Use your real, logged-in browser to automate daily tasks.

microsoft/promptflow

Build high-quality LLM apps - from prototyping, testing to production deployment and monitoring.

OpenDCAI/DataFlow

[SIGMOD'27] Easy Data Preparation with latest LLMs-based Operators and Pipelines.

ed-donner/llm_engineering

Repo to accompany my mastering LLM engineering course

jingyaogong/minimind

🧠 Train a 64M-parameter LLM from scratch in just 2h!

cactus-compute/cactus

Sub-50mb agentic, voice and vision intelligence for consumer devices.

rikkahub/rikkahub

RikkaHub is an Android APP that supports for multiple LLM providers.

THUDM/slime

slime is an LLM training framework for RL Scaling.

ZeroLu/awesome-nanobanana-pro

🚀 An awesome list of curated Nano Banana pro prompts and examples. Your go-to resource for mastering prompt engineering and exploring the creative potential of the Nano banana pro(Nano banana 2) AI im

SillyTavern/SillyTavern

LLM Frontend for Power Users.

tmc/langchaingo

LangChain for Go, the easiest way to write LLM-based programs in Go

vllm-project/vllm-ascend

Community maintained hardware plugin for vLLM on Huawei Ascend

0xPlaygrounds/rig

⚙️🦀 Build modular and scalable LLM Applications in Rust

kvcache-ai/Mooncake

Mooncake is the serving platform for Kimi, a leading LLM service provided by Moonshot AI.

llm-d/llm-d

State of the art inference performance with modern accelerators on Kubernetes

Nagi-ovo/voyager

Enhancement suite for Gemini, AI Studio, Claude, ChatGPT & DeepSeek — plus a prompt manager for any website, DeepSeek Harness included. / 面向 Gemini、AI Studio、Claude、ChatGPT 与 DeepSeek 的增强套件；其中的提示词管理器可

botpress/botpress

The open-source hub to build & deploy GPT/LLM Agents ⚡️

inkeep/open-knowledge

Beautiful, AI-native markdown IDE and LLM wiki

PaddlePaddle/PaddleOCR

Turn any PDF or image document into structured data for your AI. A powerful, lightweight OCR toolkit that bridges the gap between images/PDFs and LLMs. Supports 100+ languages.

mnfst/awesome-free-llm-apis

List of Permanent Free LLM API (API Keys)

flashinfer-ai/flashinfer

FlashInfer: Kernel Library for LLM Serving

UfoMiao/zcf

Zero-Config Code Flow for Claude code & Codex

Goochbeater/Spiritual-Spell-Red-Teaming

A repo for jailbreaking various LLMs, mainly Claude

NVIDIA/garak

the LLM vulnerability scanner

zhnt/loushang

AI-native agent harness for coding workflows by python: multi-model LLM orchestration, stateful sessions, tool governance, traceable delivery, and provider routing for GPT, Claude, DeepSeek, Qwen, K

linkedin/Liger-Kernel

Efficient Triton Kernels for LLM Training

linuxhsj/openclaw-zero-token

OpenClaw: Use All Major AI Models NO API Token! Claude/ChatGPT/Gemini/DeepSeek/Doubao/Grok/Qwen/Manus/Kimi

BlinkDL/RWKV-LM

RWKV (pronounced RwaKuv) is an RNN with great LLM performance, which can also be directly trained like a GPT transformer (parallelizable). We are at RWKV-7 "Goose". So it's combining the best of RNN a

EricLBuehler/mistral.rs

Fast, flexible LLM inference

open-compass/opencompass

OpenCompass is an LLM evaluation platform, supporting a wide range of models from OpenAI, Anthropic, Gemini, Qwen, GLM, DeepSeek, etc, across 100+ datasets covering knowledge, reasoning, coding, scien

toon-format/toon

🎒 Token-Oriented Object Notation (TOON) – compact, human-readable serialization of JSON data for LLM prompts. TypeScript SDK, CLI, benchmarks.

Zipstack/unstract

LLM-Driven Extraction of Unstructured Data — Built for API Deployments & ETL Pipeline Workflows

oobabooga/textgen

Open-source desktop app for local LLMs. Text, vision, tool-calling, OpenAI/Anthropic-compatible API. 100% private.

lemonade-sdk/lemonade

Lemonade helps users discover and run local AI apps by serving optimized LLMs right from their own GPUs and NPUs. Join our discord: https://discord.gg/5xXzkMu8Zk

InternLM/lmdeploy

LMDeploy is a toolkit for compressing, deploying, and serving LLMs.

567-labs/instructor

structured outputs for llms

NVIDIA/Model-Optimizer

A unified library of SOTA model optimization techniques like quantization, distillation, pruning, neural architecture search, speculative decoding, etc. It compresses deep learning models for downstre

MemoriLabs/Memori

Memori is agent-native memory infrastructure. A LLM-agnostic layer that turns agent execution and conversation into structured, persistent state for production systems. Built for enterprise, Memori wo

tailcallhq/forgecode

AI enabled pair programmer for Claude, GPT, O Series, Grok, Deepseek, Gemini and 300+ models

mostlygeek/llama-swap

Reliable model swapping for any local AI server. Chosen by thousands of users. Batteries included. If we don't have it, you probably don't need it.

microsoft/BitNet

Official inference framework for 1-bit LLMs

liguodongiot/llm-action

本项目旨在分享大模型相关技术原理以及实战经验（大模型工程化、大模型应用落地）

zhu1090093659/dsh-web

DeepSeek Harness (DSH) Web 插件聚合生态 · 万物皆插件，通过创意工坊分发｜｜DeepSeek Harness (DSH) Web Plugin Aggregation Ecosystem · Everything is a plugin, distributed via the Creative Workshop

promptslab/Awesome-Prompt-Engineering

This repository contains a hand-curated resources for Prompt Engineering with a focus on Generative Pre-trained Transformer (GPT), ChatGPT, PaLM etc

skyzh/tiny-llm

learn LLM inference system on Apple Silicon for systems engineers: build a tiny vLLM + Qwen

neuml/txtai

💡 All-in-one AI framework for semantic search, LLM orchestration and language model workflows

microsoft/TinyTroupe

LLM-powered multiagent persona simulation for imagination enhancement and business insights.

ShishirPatil/gorilla

Gorilla: Training and Evaluating LLMs for Function Calls (Tool Calls)

0xsline/awesome-deepseek-harness

DeepSeek Harness (DSH) ecosystem: curated plugins, tools, and infrastructure from dsh-external/hub and the public dsh-plugin topic.

bentoml/OpenLLM

Run any open-source LLMs, such as DeepSeek and Llama, as OpenAI compatible API endpoint in the cloud.

KalyanKS-NLP/llm-engineer-toolkit

A curated list of 120+ LLM libraries category wise.

voyager-crew/voyager

Enhancement suite for Gemini, AI Studio, Claude, ChatGPT & DeepSeek — plus a prompt manager for any website, DeepSeek Harness included. / 面向 Gemini、AI Studio、Claude、ChatGPT 与 DeepSeek 的增强套件；其中的提示词管理器可

georgezouq/awesome-ai-in-finance

🔬 A curated list of awesome LLMs & deep learning strategies & tools in financial market.

h2oai/h2o-llmstudio

H2O LLM Studio - a framework and no-code GUI for fine-tuning LLMs. Documentation: https://docs.h2o.ai/h2o-llmstudio/

microsoft/LLMLingua

[EMNLP'23, ACL'24] To speed up LLMs' inference and enhance LLM's perceive of key information, compress the prompt and KV-Cache, which achieves up to 20x compression with minimal performance loss.

google-deepmind/gemma

Gemma open-weight LLM library, from Google DeepMind

songquanpeng/one-api

LLM API 管理 & 分发系统，支持 OpenAI、Azure、Anthropic Claude、Google Gemini、DeepSeek、字节豆包、ChatGLM、文心一言、讯飞星火、通义千问、360 智脑、腾讯混元等主流模型，统一 API 适配，可用于 key 管理与二次分发。单可执行文件，提供 Docker 镜像，一键部署，开箱即用。LLM API management & key

nomic-ai/gpt4all

GPT4All: Run Local LLMs on Any Device. Open-source and available for commercial use.

FlagOpen/FlagEmbedding

Retrieval and Retrieval-augmented LLMs

vllm-project/llm-compressor

State-of-the-art LLM compression, built for production inference with vLLM

itayinbarr/little-coder

A harness optimized to smaller LLMs

oumi-ai/oumi

Easily fine-tune (SFT/RL), evaluate, and deploy Qwen, Gemma, or any open agentic LLM/VLM!

neo4j-labs/llm-graph-builder

Neo4j graph construction from unstructured data using LLMs

lightseekorg/tokenspeed

TokenSpeed is a speed-of-light LLM inference engine.

bentoml/BentoML

The easiest way to serve AI apps and models - Build Model Inference APIs, Job queues, LLM apps, Multi-model pipelines, and more!

kubeflow/trainer

Distributed AI Model Training and LLM Fine-Tuning on Kubernetes

yihong0618/xiaogpt

Play ChatGPT and other LLM with Xiaomi AI Speaker

mindcraft-bots/mindcraft

Minecraft AI with LLMs+Mineflayer

algorithmicsuperintelligence/optillm

Optimizing inference proxy for LLMs

bitsandbytes-foundation/bitsandbytes

Accessible large language models via k-bit quantization for PyTorch.

pytorch/ao

PyTorch native quantization for training and inference

AI-Hypercomputer/maxtext

A simple, performant, and scalable Jax LLM!

datawhalechina/happy-llm

📚 从零开始构建大模型

GaiZhenbiao/ChuanhuChatGPT

GUI for ChatGPT API and many LLMs. Supports agents, file-based QA, GPT finetuning and query with web search. All with a neat UI.

huggingface/huggingface.js

Use Hugging Face with JavaScript

huggingface/open-r1

Fully open reproduction of DeepSeek-R1

karpathy/llm-council

LLM Council works together to answer your hardest questions

ModelTC/LightLLM

LightLLM is a Python-based LLM (Large Language Model) inference and serving framework, notable for its lightweight design, easy scalability, and high-speed performance.

karpathy/llm.c

LLM training in simple, raw C/CUDA

atomicstrata/llm-wiki-compiler

The knowledge compiler. Raw sources in, interlinked wiki out. Inspired by Karpathy's LLM Wiki pattern.

deepseek-ai/awesome-deepseek-integration

Integrate the DeepSeek API into popular software

infiniflow/infinity

The AI-native database built for LLM applications, providing incredibly fast hybrid search of dense vector, sparse vector, tensor (multi-vector), and full-text.

anthropics/prompt-eng-interactive-tutorial

Anthropic's Interactive Prompt Engineering Tutorial

OvidijusParsiunas/deep-chat

Fully customizable AI chatbot component for your website

ludwig-ai/ludwig

Low-code framework for building custom LLMs, neural networks, and other AI models

jnMetaCode/agency-orchestrator

🚀 One sentence → your one-person company of AI experts → complete deliverable in minutes. 276 CN + 184 EN + 5 more languages (ko/ru/pt-BR/id/ar) · zero-code YAML · auto-verified acceptance · Web Studi

deepseek-ai/awesome-deepseek-agent

Helicone/helicone

🧊 Open source LLM observability platform. One line of code to monitor, evaluate, and experiment. YC W23 🍓

bojieli/ai-infra-book

《深入理解 AI Infra：量化分析与系统设计》（李博杰 著）开源书稿：从硬件约束和模型架构出发，量化推导 LLM 推理与训练系统设计。含全书正文、PDF、配套计算工具与实验

JohnSnowLabs/spark-nlp

State of the Art Natural Language Processing

mlabonne/llm-course

Course to get into Large Language Models (LLMs) with roadmaps and Colab notebooks.

Zafer-Liu/Data-Analysis-Agent

🚀你的私人数据分析助手。通过对话式交互，自动生成可视化报表与商业洞察，让数据决策变得像聊天一样简单。 🚀 Your personal data analysis assistant. Say goodbye to complex SQL and Excel formulas. An LLM-powered data analysis agent. Chat with your data to

langroid/langroid

Harness LLMs with Multi-Agent Programming

stanford-oval/storm

An LLM-powered knowledge curation system that researches a topic and generates a full-length report with citations.

GeeeekExplorer/nano-vllm

Nano vLLM

ai-boost/awesome-prompts

Curated prompts, frameworks and papers for LLMs and agents, with an engineering bias. Prompt engineering, context engineering, agent harnesses, red teaming.

deer-flow/llm-space

A desktop app to prototype agent ideas, inspect every harness step, replay failures, and evaluate performance, all in one place. Local-first, cloud-ready for managed agents.

openai/chatgpt-retrieval-plugin

The ChatGPT Retrieval Plugin lets you easily find personal or work documents by asking questions in natural language.

rllm-org/rllm

Democratizing Reinforcement Learning for LLMs

OpenMOSS/MOSS

An open-source, tool-augmented conversational language model from Fudan University

Hannibal046/Awesome-LLM

Awesome-LLM: a curated list of Large Language Model

google/XNNPACK

High-efficiency floating-point neural network inference operators for mobile, server, and Web

FireRedTeam/FireRed-OpenStoryline

FireRed-OpenStoryline is an AI video editing agent that transforms manual editing into intention-driven directing through natural language interaction, LLM-powered planning, and precise tool orchestra

waybarrios/vllm-mlx

High-performance OpenAI and Anthropic compatible LLM inference server for Apple Silicon. Native MLX, continuous batching, multimodal models, MCP tool calling, and Claude Code support.

mufeedvh/code2prompt

A CLI tool to convert your codebase into a single LLM prompt with source tree, prompt templating, and token counting.

pezzolabs/pezzo

🕹️ Open-source, developer-first LLMOps platform designed to streamline prompt design, version management, instant delivery, collaboration, troubleshooting, observability and more.

microsoft/LMOps

General technology for enabling AI capabilities w/ LLMs and MLLMs

LazyAGI/LazyLLM

Easiest and laziest way for building multi-agent LLMs applications.

AlexAnys/awesome-openclaw-usecases-zh

🇨🇳 OpenClaw中文用例大全 | 50个真实场景 | 国内特色 + 海外案例的国内适配 | 自动化办公·内容创作·运维·AI助理·知识管理 | 新手友好

openlit/openlit

Same agent Smaller bill. The tools and infrastructure to get more work from your AI. Fewer tokens. Less compute. Lower cost.

The-Pocket/PocketFlow

Pocket Flow: 100-line LLM framework. Let Agents build Agents!

SwanHubX/SwanLab

⚡️SwanLab - an open-source, modern-design AI training tracking and visualization tool. Supports Cloud / Self-hosted use. Integrated with PyTorch / Transformers / verl / LLaMA Factory / ms-swift / Ultr

PaddlePaddle/PaddleFormers

PaddleFormers is an easy-to-use library of pre-trained large language model zoo based on PaddlePaddle.

FareedKhan-dev/train-llm-from-scratch

A straightforward method for training your LLM, from downloading data to generating text.

envoyproxy/ai-gateway

Manages Unified Access to Generative AI Services built on Envoy Gateway

qualcomm/aimet

AIMET is a library that provides advanced quantization and compression techniques for trained neural network models.

PaddlePaddle/FastDeploy

High-performance Inference and Deployment Toolkit for LLMs and VLMs based on PaddlePaddle

MeteorNOX/DeepSeek-Balance-Whale-Widget

DeepSeek Harness（DSH）一只住在 DSH 界面右下角的小鲸鱼娘，帮你盯着DeepSeek账户余额。QQ弹弹，支持拖拽吸附、左吸附翻转、数字滚动动画，随界面自动启用，建议直接喊来你的dsh安装

OpenCSGs/csghub

CSGHub is a brand-new open-source platform for managing LLMs, developed by the OpenCSG team. It offers both open-source and on-premise/SaaS solutions, with features comparable to Hugging Face. Gain fu

ZJU-LLMs/Foundations-of-LLMs

A book for Learning the Foundations of LLMs

zgsm-ai/costrict

Costrict - strict AI coder for enterprises, quality first, including AI Agent, AI CodeReview, AI Completion.

DataTalksClub/llm-zoomcamp

LLM Zoomcamp - a free online course about real-life applications of LLMs. In 10 weeks you will learn how to build an AI system that answers questions about your knowledge base. Register here 👇🏼

mozilla-ai/any-llm

Communicate with an LLM provider using a single interface

syv-ai/qwen38-27b-rtx3090

Qwen3.8-27B on a single RTX 3090 with vLLM: ~1,000 tok/s at 64 concurrent (int8 tensor-core GEMMs, fp16 DeltaNet state), ~114 tok/s single-user at default sampling / ~124 greedy (MTP drafts, own-outpu

whiteguo233/OpenBiliClaw

本地私有、开源的自进化跨平台 AI 内容发现 Agent：先理解你，再主动从 B站、小红书、抖音、YouTube、X、知乎、Reddit、微博等平台与开放 Web 寻找内容。（支持 deepseek harness 插件） | Local-first open-source cross-platform AI content discovery agent: understands you, th

PaddlePaddle/PaddleNLP

Easy-to-use and powerful LLM and SLM library with awesome model zoo.

vibrantlabsai/ragas

Supercharge Your LLM Application Evaluations 🚀

Lordog/dive-into-llms

《动手学大模型Dive into LLMs》系列编程实践教程

HKUDS/AutoAgent

"AutoAgent: Fully-Automated and Zero-Code LLM Agent Framework"

HUANGCHIHHUNGLeo/claude-real-video

Let Claude (or any LLM) actually watch a video — scene-aware, deduplicated frames + transcript, from a URL or local file. Runs locally, MIT.

zilliztech/GPTCache

Semantic cache for LLMs. Fully integrated with LangChain and llama_index.

zai-org/ChatGLM2-6B

ChatGLM2-6B: An Open Bilingual Chat LLM | 开源双语对话语言模型

NVIDIA-NeMo/Curator

Scalable data pre processing and curation toolkit for LLMs

qinshihu/itops-agent-platform

China's 1st enterprise multi-agent IT ops platform. LLM-powered auto-remediation for Zabbix/Prometheus. Docker deploy.

latitude-dev/latitude-llm

Open-source observability for AI agents. Find where your agents fail, dispatch your coding agent to fix it, and verify the fix against real traces.

icereed/paperless-gpt

Use LLMs and LLM Vision (OCR) to handle paperless-ngx - Document Digitalization powered by AI

humanlayer/12-factor-agents

What are the principles we can use to build LLM-powered software that is actually good enough to put in the hands of production customers?

OpenBMB/XAgent

An Autonomous LLM Agent for Complex Task Solving

JushBJJ/Mr.-Ranedeer-AI-Tutor

A GPT-4 AI Tutor Prompt for customizable personalized learning experiences.

NirDiamant/Prompt_Engineering

22 prompt engineering techniques with hands-on Jupyter Notebook tutorials, from fundamental concepts to advanced strategies for leveraging LLMs.

QwenLM/Qwen

The official repo of Qwen (通义千问) chat & pretrained large language model proposed by Alibaba Cloud.

vllm-project/guidellm

Evaluate and Enhance Your LLM Deployments for Real-World Inference Needs

marcelroed/gigatoken

Language model tokenization at GB/s

kuafuai/DevOpsGPT

Multi agent system for AI-driven software development. Combine LLM with DevOps tools to convert natural language requirements into working software. Supports any development language and extends the e

linto-ai/whisper-timestamped

Multilingual Automatic Speech Recognition with word-level timestamps and confidence

ucbepic/docetl

A system for agentic LLM-powered data processing and ETL

alibaba/rtp-llm

RTP-LLM: Alibaba's high-performance LLM inference engine for diverse applications.

BlinkDL/ChatRWKV

ChatRWKV is like ChatGPT but powered by RWKV (100% RNN) language model, and open source.

xiaobright/dsh-anchored-standard

Two-phase DeepSeek Harness preset: Minimal-aligned bootstrap, then full Standard tools (Project2 98/99)

datawhalechina/llms-from-scratch-cn

仅需Python基础，从0构建大语言模型；从0逐步构建GLM4\Llama3\RWKV6， 深入理解大模型原理

lucidrains/vector-quantize-pytorch

Vector (and Scalar) Quantization, in Pytorch

mishushakov/llm-scraper

Turn any webpage into structured data using LLMs

rafska/awesome-local-llm

A curated list of awesome platforms, tools, practices and resources that helps run LLMs locally

theJayTea/WritingTools

The world's smartest system-wide grammar assistant; a better version of the Apple Intelligence Writing Tools. Works on Windows, Linux, & macOS, with the free Gemini API, local LLMs, & more.

data-infra/cube-studio

cubestudio开源云原生一站式机器学习/深度学习/大模型AI平台/MaaS/mlops/人工智能平台/训推平台，算法全链路流程，多租户，算力租赁平台，token中转，拖拉拽任务流pipeline编排，多机多卡分布式训练，超参搜索，推理服务，VGPU虚拟化，云边端协同，边缘计算，自动化标注平台，deepseek等大模型sft微调/奖励模型/强化学习训练，vllm/ollama/mindie大模

ax-llm/ax

The pretty much "official" DSPy framework for Typescript

datawhalechina/llm-universe

本项目是一个面向小白开发者的大模型应用开发教程，在线阅读地址：https://datawhalechina.github.io/llm-universe/

bytebase/dbhub

Token conscious database MCP server for Postgres, MySQL, SQL Server, MariaDB, SQLite.

wdndev/llm_interview_note

主要记录大语言大模型（LLMs） 算法（应用）工程师相关的知识及面试题

AiHubCN/Awesome-Chinese-LLM

整理开源的中文大语言模型，以规模较小、可私有化部署、训练成本较低的模型为主，包括底座模型，垂直领域微调及应用，数据集与教程等。

Xilinx/brevitas

Brevitas: neural network quantization in PyTorch

huggingface/course

The Hugging Face course on Transformers

opensolon/solon

🔥 Java enterprise application development framework for full scenario: Restrained, Efficient, Open, Ecologicalll!!! 700% higher concurrency 50% memory savings Startup is 10 times faster. Packing 90% s

Tiiny-AI/PowerInfer

High-speed Large Language Model Serving for Local Deployment

Rizzo-AI-Academy/rizzo-pii

Local-first privacy guard: anonymize your documents before sharing with LLMs.

ggml-org/llama.vscode

VS Code extension for LLM-assisted code/text completion

LuckyOne7777/LLM-Trading-Lab

This repo powers my experiment where ChatGPT manages a real-money micro-cap stock portfolio.

QwenLM/Qwen3.8

Qwen3.8 is the large language model series developed by Qwen team, Alibaba Group.

datawhalechina/llm-cookbook

面向开发者的 LLM 入门教程，吴恩达大模型系列课程中文版

intel/auto-round

PTQ-Level Efficiency, QAT-Level Accuracy | 训练后量化的效率，量化感知训练的精度

raiyanyahya/how-to-train-your-gpt

Build a modern LLM from scratch. Every line commented. Explained like we are five.

R6410418/Jackrong-llm-finetuning-guide

jamesob/local-llm

Everything I know about running LLMs locally

agentuniverse-ai/agentUniverse

agentUniverse is a LLM multi-agent framework that allows developers to easily build multi-agent applications.

matt1398/claude-devtools

The missing DevTools for Claude Code — inspect session logs, tool calls, token usage, subagents, and context window in a visual UI. Free, open source.

bigscience-workshop/petals

🌸 Run LLMs at home, BitTorrent-style. Fine-tuning and inference up to 10x faster than offloading

ymcui/Chinese-LLaMA-Alpaca

中文LLaMA&Alpaca大语言模型+本地CPU/GPU训练部署 (Chinese LLaMA & Alpaca LLMs)

theopenco/llmgateway

Route, manage, and analyze your LLM requests across multiple providers with a unified API interface.

Exorust/TorchLeet

LeetCode for PyTorch — 65 ML/AI interview problems from real interviews at Google, Meta, Anthropic. Jupyter notebooks, an auto-grader, and an MCP AI tutor.

win4r/AISuperDomain

Aila(AI超元域): The premier AI integration tool for Windows, macOS, and Android. Ask once, get answers from 10+ AIs like ChatGPT, Gemini, Claude3, Copilot, Poe, perplexity and more. Features customizable

skindhu/Build-A-Large-Language-Model-CN

《Build a Large Language Model (From Scratch)》是一本深入探讨大语言模型原理与实现的电子书，适合希望深入了解 GPT 等大模型架构、训练过程及应用开发的学习者。为了让更多中文读者能够接触到这本极具价值的教材，我决定将其翻译成中文，并通过 GitHub 进行开源共享。

AgentOps-AI/agentops

Python SDK for AI agent monitoring, LLM cost tracking, benchmarking, and more. Integrates with most LLMs and agent frameworks including CrewAI, Agno, OpenAI Agents SDK, Langchain, Autogen, AG2, and Ca

intel/neural-compressor

SOTA low-bit LLM quantization (INT8/FP8/MXFP8/INT4/MXFP4/NVFP4) & sparsity; leading model compression techniques on PyTorch, TensorFlow, and ONNX Runtime

yuruotong1/autoMate

Like Manus, Computer Use Agent(CUA) and Omniparser, we are computer-using agents.AI-driven local automation assistant that uses natural language to make computers work by themselves

CodeSoul-co/Hypha

Harness-oriented agent system framework for production-grade LLM agent applications

ModelCloud/GPTQModel

LLM model quantization (compression) toolkit with HW acceleration support for Nvidia (CUDA), AMD (ROCm), Intel GPU (XPU), Huawei Ascent (NPU), and Intel/AMD/Apple CPU via HF, vLLM, and SGLang.

eugeneyan/open-llms

📋 A list of open LLMs available for commercial use.

bbycroft/llm-viz

3D Visualization of an GPT-style LLM

fnnx-ai/scikit-llm

Seamlessly integrate LLMs into scikit-learn.

OpenGVLab/Ask-Anything

[CVPR2024 Highlight][VideoChatGPT] ChatGPT with video understanding! And many more supported LMs such as miniGPT4, StableLM, and MOSS.

heshengtao/comfyui_LLM_party

LLM Agent Framework in ComfyUI includes MCP sever, Omost,GPT-sovits, ChatTTS,GOT-OCR2.0, and FLUX prompt nodes,access to Feishu,discord,and adapts to all llms with similar openai / aisuite interfaces,

ThilinaRajapakse/simpletransformers

Transformers for Information Retrieval, Text Classification, NER, QA, Language Modelling, Language Generation, T5, Multi-Modal, and Conversational AI

databrickslabs/dolly

Databricks’ Dolly, a large language model trained on the Databricks Machine Learning Platform

NVIDIA/kvpress

LLM KV cache compression made easy

tensorflow/model-optimization

A toolkit to optimize ML models for deployment for Keras and TensorFlow, including quantization and pruning.

vibeinging/deepseek-harness-desktop-app

DeepSeek Harness Desktop App: a local AI desktop workspace for DSH Sessions, projects, files, web research, plugins, and Office artifacts.

llm-workflow-engine/llm-workflow-engine

Power CLI and Workflow manager for LLMs (core package)

notatestuser/ds4-control

macOS menubar app for fast local DeepSeek V4.1, with 1M context.

wgwang/awesome-LLMs-In-China

中国大模型

nndl/llm-beginner

《大模型与智能体》电子书与 6 个编程任务：Transformer、mini-GPT、SFT/DPO、RAG、工具调用与编程智能体。

changyeyu/LLM-RL-Visualized

🌟100+ 原创 LLM / RL 原理图📚，《大模型算法》作者巨献！💥（100+ LLM/RL Algorithm Maps ）

llm-attacks/llm-attacks

Universal and Transferable Attacks on Aligned Language Models

claw-eval/claw-eval

Claw-Eval is an evaluation harness for evaluating LLM as agents. All tasks verified by humans.

InternLM/lagent

A lightweight framework for building LLM-based agents

azrtydxb/Fastllm-proxy

The lowest-overhead LLM router. Production-ready, highly available, one OpenAI-compatible endpoint in front of 80 providers and your own vLLM/SGLang — 0.76 µs per request, no I/O on the request path,

Lum1104/dsh-browser

Chrome sidebar extension that lets DeepSeek Harness operate your browser directly, no vision capabilities required. 一款 Chrome 侧边栏扩展程序，可让 DeepSeek Harness 直接操控您的浏览器，无需视觉能力。

mixelpixx/Konnect

AI-assisted PCB design for KiCAD 10. Native KiCAD plugin — a single Rust binary exposing 217 schematic, layout, routing, placement, design-review, and manufacturing tools to Claude, or the LLM of your

Electricitysheep/dsh-handbook

DeepSeek Harness (dsh) 从 0 到 1 深度手册：安装/插件开发/性能调优/实测案例/同模型多 Agent 实测对比（中文 + 英文 PDF）

QwenAudio/Fun-ASR

Fun-ASR speech recognition models, with native Hugging Face Transformers support for Fun-ASR-Nano and separate FunASR, vLLM and llama.cpp deployment paths.

yangjianxin1/Firefly

Firefly: 大模型训练工具，支持训练Qwen2.5、Qwen2、Yi1.5、Phi-3、Llama3、Gemma、MiniCPM、Yi、Deepseek、Orion、Xverse、Mixtral-8x7B、Zephyr、Mistral、Baichuan2、Llma2、Llama、Qwen、Baichuan、ChatGLM2、InternLM、Ziya2、Vicuna、Bloom等大模型

trigaten/Learn_Prompting

Prompt Engineering, Generative AI, and LLM Guide by Learn Prompting | Join our discord for the largest Prompt Engineering learning community

cvg/LightGlue

LightGlue: Local Feature Matching at Light Speed (ICCV 2023)

Mooler0410/LLMsPracticalGuide

A curated list of practical guide resources of LLMs (LLMs Tree, Examples, Papers)

EricLBuehler/candle-vllm

Efficent platform for inference and serving local LLMs including an OpenAI compatible API server.

aingdesk/AingDesk

AingDesk是一款简单好用的AI助手，支持知识库、模型API、分享、联网搜索、智能体，它还在飞快成长中。 AingDesk is a simple and easy-to-use AI assistant that supports knowledge bases, model APIs, sharing, internet search, and intelligent agents. It

Dryxio/auto-re-agent

Open-source AI reverse-engineering agent using Ghidra and LLMs to reconstruct and validate C/C++ functions from binaries.

predibase/lorax

Multi-LoRA inference server that scales to 1000s of fine-tuned LLMs

warpfront/hipfire

RDNA-native LLM inference engine in Rust.

promptslab/Promptify

Prompt Engineering | Prompt Versioning | Use GPT or other prompt based models to get structured output. Join our discord for Prompt-Engineering, LLMs and other latest research

EwingYangs/awesome-open-gpt

Collection of Open Source Projects Related to GPT，GPT相关开源项目合集🚀、精选🔥🔥

nottelabs/notte

Best cloud browser infrastructure and web automation platform for your AI (Coding) agents!

ggozad/oterm

the terminal client for LLMs

salesforce/TransmogrifAI

TransmogrifAI (pronounced trăns-mŏgˈrə-fī) is an AutoML library for building modular, reusable, strongly typed machine learning workflows on Apache Spark with minimal hand-tuning

ashishps1/learn-ai-engineering

Learn AI and LLMs from scratch using free resources

RightNow-AI/picolm

Run a 1-billion parameter LLM on a $10 board with 256MB RAM

Mirrowel/LLM-API-Key-Proxy

Universal LLM Gateway: One API, every LLM. OpenAI/Anthropic-compatible endpoints with multi-provider translation and intelligent load-balancing.

ttttccxxui/DataInfra-RedactionEverything

DataInfra Series. Redact EVERYTHING with local llms and vlms.

askimo-ai/askimo

AI Client for chat, RAG, plans, MCP tools, and agents. Support multiple LLMs (Anthropic, OpenAI, VertexAI, vLLM, Nvidia NIM, Gemini, OpenRouter)

vince-lam/awesome-local-llms

Compare compatible open-source local LLM projects by their metrics to assess popularity and activeness.

Anil-matcha/awesome-dsh-plugin

A curated list of plugins for DeepSeek Harness (dsh) - DeepSeek Harness plugin ecosystem

huawei-noah/Pretrained-Language-Model

Pretrained language model and its related optimization techniques developed by Huawei Noah's Ark Lab.

yuanzhoulvpi2017/zero_nlp

中文nlp解决方案(大模型、数据、模型、训练、推理)

vndee/llm-sandbox

Lightweight and portable LLM sandbox runtime (code interpreter) Python library.

mnemon-dev/mnemon

LLM-supervised persistent memory for AI agents — graph-based recall, cross-session knowledge, single binary. Works with DeepSeek Harness, Claude Code, OpenClaw, and any agent runtime.

rashadphz/farfalle

🔍 AI search engine - self-host with local or cloud LLMs

NirDiamant/Agent_Memory_Techniques

Agent memory for LLMs: 30 runnable Jupyter notebooks covering conversation buffers, vector stores, knowledge graphs, episodic and semantic memory, MemGPT, Mem0, Letta, Zep, Graphiti, LoCoMo benchmarks

SonySemiconductorSolutions/mct-model-optimization

Model Compression Toolkit (MCT) is an open source project for neural network model optimization under efficient, constrained hardware. This project provides researchers, developers, and engineers adva

ciddwd/overlay-translator

无需 ROOT 的开源 Android 屏幕实时翻译工具，适合游戏、视觉小说和漫画。支持端侧与云端 OCR、离线 LLM、多种翻译服务和文字朗读（TTS），译文可直接显示在画面上。Open-source no-root Android real-time screen translator for games, visual novels, and manga. Supports on-devic

myICOR/myPKA

AI-powered Personal Knowledge Assistance in a folder. Built on the ICOR methodology. Plain markdown. Any LLM. Yours forever.

xenova/whisper-web

ML-powered speech recognition directly in your browser

yakami129/VirtualWife

VirtualWife是一个虚拟数字人项目，支持B站直播，支持openai、ollama

lucasjinreal/yolov7_d2

🔥🔥🔥🔥 (Earlier YOLOv7 not official one) YOLO with Transformers and Instance Segmentation, with TensorRT acceleration! 🔥🔥🔥

prism-php/prism

A unified interface for working with LLMs in Laravel

adapter-hub/adapters

A Unified Library for Parameter-Efficient and Modular Transfer Learning

teddylee777/langchain-kr

LangChain 공식 Document, Cookbook, 그 밖의 실용 예제를 바탕으로 작성한 한국어 튜토리얼입니다. 본 튜토리얼을 통해 LangChain을 더 쉽고 효과적으로 사용하는 방법을 배울 수 있습니다.

tobegit3hub/tensorflow_template_application

TensorFlow template application for deep learning

lucidrains/musiclm-pytorch

Implementation of MusicLM, Google's new SOTA model for music generation using attention networks, in Pytorch

IntelligenzaArtificiale/Free-Auto-GPT

Free Auto GPT with NO paids API is a repository that offers a simple version of Auto GPT, an autonomous AI agent capable of performing tasks independently. Unlike other versions, our implementation do

ashishpatel26/LLM-Finetuning

LLM Finetuning with peft

sauravpanda/BrowserAI

Run local LLMs like llama, deepseek-distill, kokoro and more inside your browser

666DZY666/micronet

micronet, a model compression and deploy lib. compression: 1、quantization: quantization-aware-training(QAT), High-Bit(>2b)(DoReFa/Quantization and Training of Neural Networks for Efficient Integer-Ari

gmpetrov/databerry

The no-code platform for building custom LLM Agents

alfredfrancis/ai-chatbot-framework

A python chatbot framework with Natural Language Understanding and Artificial Intelligence.

lucidrains/audiolm-pytorch

Implementation of AudioLM, a SOTA Language Modeling Approach to Audio Generation out of Google Research, in Pytorch

bentrevett/pytorch-sentiment-analysis

Tutorials on getting started with PyTorch and TorchText for sentiment analysis.

AgibotTech/agibot_x1_infer

The inference module for AgiBot X1.

addyosmani/chatty

ChattyUI - your private AI chat for running LLMs in the browser

YiVal/YiVal

Your Automatic Prompt Engineering Assistant for GenAI Applications

EgoAlpha/prompt-in-context-learning

Awesome resources for in-context learning and prompt engineering: Mastery of the LLMs such as ChatGPT, GPT-3, and FlanT5, with up-to-date and cutting-edge updates.

VainF/Torch-Pruning

[CVPR 2023] DepGraph: Towards Any Structural Pruning; LLMs, Vision Foundation Models, etc.

fiatrete/OpenDAN-Personal-AI-OS

OpenDAN is an open source Personal AI OS , which consolidates various AI modules in one place for your personal use.

OpenGVLab/InternGPT

InternGPT (iGPT) is an open source demo platform where you can easily showcase your AI models. Now it supports DragGAN, ChatGPT, ImageBind, multimodal chat like GPT-4, SAM, interactive image editing,

friuns2/Leaked-GPTs

Leaked GPTs Prompts Bypass the 25 message limit or to try out GPTs without a Plus subscription.

ai-forever/ru-gpts

Russian GPT3 models.

paulpierre/RasaGPT

💬 RasaGPT is the first headless LLM chatbot platform built on top of Rasa and Langchain. Built w/ Rasa, FastAPI, Langchain, LlamaIndex, SQLModel, pgvector, ngrok, telegram

HarderThenHarder/transformers_tasks

⭐️ NLP Algorithms with transformers lib. Supporting Text-Classification, Text-Generation, Information-Extraction, Text-Matching, RLHF, SFT etc.

llm-jp/awesome-japanese-llm

日本語LLMまとめ - Overview of Japanese LLMs

0hq/WebGPT

Run GPT model on the browser with WebGPU. An implementation of GPT inference in less than ~1500 lines of vanilla Javascript.

KimGLee/Cambium

Governance standard and reference toolset for LLM-maintained knowledge corpora

huggingface/transformers.js-examples

A collection of 🤗 Transformers.js demos and example applications

wuwangzhang1216/abliterix

Automated alignment adjustment for LLMs — direct steering, LoRA, and MoE expert-granular abliteration, optimized via multi-objective Optuna TPE.

semanser/codel

✨ Fully autonomous AI Agent that can perform complicated tasks and projects using terminal, browser, and editor.

alibaba/EasyNLP

EasyNLP: A Comprehensive and Easy-to-use NLP Toolkit

zyds/transformers-code

手把手带你实战 Huggingface Transformers 课程视频同步更新在B站与YouTube

xusenlinzy/api-for-open-llm

Openai style api for open large language models, using LLMs just as chatgpt! Support for LLaMA, LLaMA-2, BLOOM, Falcon, Baichuan, Qwen, Xverse, SqlCoder, CodeLLaMA, ChatGLM, ChatGLM2, ChatGLM3 etc. 开源

kaito-project/aikit

🏗️ Fine-tune, build, and deploy open-source LLMs easily!

huggingface/optimum-quanto

A pytorch quantization backend for optimum

hyp1231/awesome-llm-powered-agent

Awesome things about LLM-powered agents. Papers / Repos / Blogs / ...

gofireflyio/aiac

Artificial Intelligence Infrastructure-as-Code Generator.

NVlabs/MambaVision

[CVPR 2025] Official PyTorch Implementation of MambaVision: A Hybrid Mamba-Transformer Vision Backbone

nicepkg/aide

Conquer Any Code in VSCode: One-Click Comments, Conversions, UI-to-Code, and AI Batch Processing of Files! 在 VSCode 中征服任何代码：一键注释、转换、UI 图生成代码、AI 批量处理文件！💪

harleyszhang/llm_note

LLM notes, including model inference, transformer model structure, and llm framework code analysis notes.

JIA-Lab-research/LISA

Project Page for "LISA: Reasoning Segmentation via Large Language Model"

FujitsuResearch/OneCompression

Python package for LLM compression

OpenPPL/ppq

PPL Quantization Tool (PPQ) is a powerful offline neural network quantization tool.

lucidrains/reformer-pytorch

Reformer, the efficient Transformer, in Pytorch

THUDM/LongWriter

[ICLR 2025] LongWriter: Unleashing 10,000+ Word Generation from Long Context LLMs

kimtth/azure-openai-llm-notes

A curated collection of resources for 🌌 Azure OpenAI, 🦙 LLMs (+RAG, Agents). Monthly Updates.

refuel-ai/autolabel

Label, clean and enrich text datasets with LLMs.

aryn-ai/sycamore

🍁 Sycamore is an LLM-powered search and analytics platform for unstructured data.

dot-agent/nextpy

🤖Self-Modifying Framework from the Future 🔮 World's First AMS

jmaczan/tiny-vllm

Build your own high performance LLM inference engine in C++ and CUDA - a smaller version of vLLM

deepspeedai/DeepSpeed-MII

MII makes low-latency and high-throughput inference possible, powered by DeepSpeed.

AI-Efficiency/Awesome-Model-Quantization

A curated collection of papers, benchmarks, surveys, and tools for model quantization, covering low-bit networks, LLMs, multimodal and generative models, vector and lattice quantization, and efficient

christianversloot/machine-learning-articles

🧠💬 Articles I wrote about machine learning, archived from MachineCurve.com.

chenking2020/FindTheChatGPTer

ChatGPT爆火，开启了通往AGI的关键一步，本项目旨在汇总那些ChatGPT的开源平替们，包括文本大模型、多模态大模型等，为大家提供一些便利

dineshsoudagar/local-llms-on-android

Run local LLMs like Gemma, Qwen, and LLaMA on Android for offline, private, real-time chat and question answering with LiteRT and ONNX Runtime.

ImageOptim/libimagequant

Palette quantization library that powers pngquant and other PNG optimizers

sums001/Deepseek-API

Reverse engineered Deepseek chat into an OpenAI compatible API. Access V4 and R1 models through a simple REST interface without API keys or billing.

business-science/awesome-generative-ai-data-scientist

A curated list of 100+ resources for building and deploying generative AI specifically focusing on helping you become a Generative AI Data Scientist with LLMs

gary23w/nl-veil

Open-source AI coding desktop app with parallel agents and persistent project memory. Local models, Ollama or cloud AI. Windows, macOS and Linux.

huawei-csl/KVarN

KVarN is a native vLLM KV-cache quantization backend for your agents: 3-5x more context, throughput above FP16, and FP16-level accuracy. Calibration-free, one flag.

Blackwellboy/model-serving-minefield

Community registry of LLM serving-path traps that produce confidently wrong measurements: templates, tool parsers, reasoning fields, quant kernel paths, CUDA toolchains, KV allocation, eval harnesses,

LycheeMem/LycheeMem

Lightweight Long-Term Memory for LLM Agents.

Alpha-VLLM/Lumina-T2X

Lumina-T2X is a unified framework for Text to Any Modality Generation

juxhinr/bindwidth

Evidence-aware on-prem LLM inference sizing and TCO calculator

buaacyw/MeshAnything

[ICLR 2025] From anything to mesh like human artists. Official impl. of "MeshAnything: Artist-Created Mesh Generation with Autoregressive Transformers"

EmbeddedLLM/JamAIBase

The collaborative spreadsheet for AI. Chain cells into powerful pipelines, experiment with prompts and models, and evaluate LLM responses in real-time. Work together seamlessly to build and iterate on

gameworkerkim/vibe-investing

AI-powered Vibe Investing for NASDAQ, S&P500 & crypto: LLM quant trading tools, multi-agent backtesting, and data-driven market columns (mNAV arbitrage, BTC-Nasdaq coupling, Alpha Arena). 미국 주식·가상화폐 A

sherlockchou86/VideoPipe

A cross-platform video structuring (video analysis) framework based on CV models & mLLM.

RManLuo/Awesome-LLM-KG

Awesome papers about unifying LLMs and KGs

relari-ai/continuous-eval

Data-Driven Evaluation for LLM-Powered Applications

jjiantong/Awesome-KV-Cache-Optimization

[ACL 2026] Towards Efficient Large Language Model Serving: A Survey on System-Aware KV Cache Optimization

qingsongedu/time-series-transformers-review

A professionally curated list of awesome resources (paper, code, data, etc.) on transformers in time series.

sobelio/llm-chain

`llm-chain` is a powerful rust crate for building chains in large language models allowing you to summarise text and complete complex tasks

NVIDIA/NeMo-Relay

Multi-language agent runtime and library for execution scope management, lifecycle events, and middleware on tool and LLM calls.

RWKV/rwkv.cpp

INT4/INT5/INT8 and FP16 inference on CPU for RWKV language model

NightMean/OlliteRT

Turn your Android phone into an OpenAI-compatible LLM inference server - Fully local, private and Open Source

hardrave/NIGHTRUN

Boot your PC straight into an LLM. Rust, UEFI-resident, no operating system underneath.

PrithivirajDamodaran/FlashRank

Lite & Super-fast re-ranking for your search & retrieval pipelines. Supports SoTA Listwise and Pairwise reranking based on LLMs and cross-encoders and more. Created by Prithivi Da, open for PRs & C

huggingface/llm-ls

LSP server leveraging LLMs for code completion (and more?)

streamcoreai/streamcore-server

Open-source realtime voice agent server in Go with WebRTC (WHIP), barge-in, streaming STT/LLM/TTS pipelines, plugin system, multi-language SDKs, SIP telephony, ESP32 support & fully local mode.

thushan/olla

High-performance lightweight proxy and load balancer for LLM infrastructure. Intelligent routing, automatic failover and unified model discovery across local and remote inference backends.

pzqpzq/LSF_MDia

[ICML 2026] Let LLMs invent and evolve languages for efficient reasoning.

AEON-7/Qwen3.6-27B-AEON-Ultimate-Uncensored-DFlash

Fully uncensored, capability-enhanced abliteration of Qwen3.6-27B. NVFP4 + z-lab DFlash speculative decoding (n=12) on the unified ghcr.io/aeon-7/aeon-vllm-ultimate:latest container, tuned for long-co

HarryR/z80ai

Z80-μLM is a 2-bit quantized language model small enough to run on an 8-bit Z80 processor. Train conversational models in Python, export them as CP/M .COM binaries, and chat with your vintage computer

X-LANCE/SLAM-LLM

A Framework for Speech, Language, Audio, Music Processing with Large Language Model

pgalko/BambooAI

A Python library powered by Language Models (LLMs) for conversational data discovery and analysis.

oxbshw/LLM-Agents-Ecosystem-Handbook

One-stop handbook for building, deploying, and understanding LLM agents with 60+ skeletons, tutorials, ecosystem guides, and evaluation tools.

Farama-Foundation/ChatArena

ChatArena (or Chat Arena) is a Multi-Agent Language Game Environments for LLMs. The goal is to develop communication and collaboration capabilities of AIs.

georgian-io/LLM-Finetuning-Toolkit

Toolkit for fine-tuning, ablating and unit-testing open-source LLMs.

kaqijiang/Auto-GPT-ZH

Auto-GPT中文版本及爱好者组织 同步更新原项目 AI领域创业 自媒体组织 用AI工作学习创作变现

shenmintao/marginalia

A library-science-inspired personal knowledge management system with LLM agents

BjornMelin/docmind-ai-llm

DocMind AI is a powerful, open-source Streamlit application leveraging LlamaIndex, LangGraph, and local Large Language Models (LLMs) via Ollama, LMStudio, llama.cpp, or vLLM for advanced document anal

stochasticai/xTuring

Build, personalize and control your own LLMs. From data pre-processing to fine-tuning, xTuring provides an easy way to personalize open-source LLMs. Join our discord community: https://discord.gg/TgHX

AGI-Edgerunners/LLM-Adapters

Code for our EMNLP 2023 Paper: "LLM-Adapters: An Adapter Family for Parameter-Efficient Fine-Tuning of Large Language Models"

GoogleCloudPlatform/terraform-genai-knowledge-base

Fine tune an LLM model to answer questions from your documents.

KwaiKEG/KwaiAgents

A generalized information-seeking agent system with Large Language Models (LLMs).

SeekingDream/Static-to-Dynamic-LLMEval

The official GitHub repository of the paper "Recent advances in large language model benchmarks against data contamination: From static to dynamic evaluation"

huawei-csl/SINQ

Welcome to the official repository of SINQ! A novel, fast and high-quality quantization method designed to make any Large Language Model smaller while preserving accuracy [ICML 2026]

IST-DASLab/marlin

FP16xINT4 LLM inference kernel that can achieve near-ideal ~4x speedups up to medium batchsizes of 16-32 tokens.

johnsonhk88/AI-Bank-Statement-Document-Automation-By-LLM-And-Personal-Finanical-Analysis-Prediction

AI Bank Statement Document Automation By LLM model and Personal Finanical Analysis

llmsresearch/llm-flashcards

211 free visual cards about large language models, from tokenization to deployment. Full collection: 376 cards.

NVlabs/alpamayo-recipes

Developer Hub for NVIDIA Alpamayo, containing ready-to-use recipes for fine-tuning, reinforcement-learning post-training, quantization, and deployment.

Sphere-AI-Lab/orbit

Stable and Efficient Reinforcement Learning for Trillion-Parameter LLMs

OpenGVLab/OmniQuant

[ICLR2024 spotlight] OmniQuant is a simple and powerful quantization technique for LLMs.

milanglacier/minuet-ai.el

💃 Dance with LLM in Your Code. Minuet offers code completion as-you-type from popular LLMs including OpenAI, Gemini, Claude, Ollama, Llama.cpp, Codestral, and more.

google/qkeras

QKeras: a quantization deep learning library for Tensorflow Keras

efeslab/Nanoflow

A throughput-oriented high-performance serving framework for LLMs

louisfb01/start-llms

A complete guide to start and improve your LLM skills in 2026 with little background in the field and stay up-to-date with the latest news and state-of-the-art techniques!

ModelTC/LightCompress

[EMNLP 2024 & AAAI 2026] A powerful toolkit for compressing large models including LLMs, VLMs, and video generative models.

hollobit/GenAI_LLM_timeline

ChatGPT, GenerativeAI and LLMs Timeline

RahulSChand/gpu_poor

Calculate token/s & GPU memory requirement for any LLM. Supports llama.cpp/ggml/bnb/QLoRA quantization

quantumaikr/quant.cpp

LLM inference with 7x longer context. Pure C, zero dependencies. Lossless KV cache compression + single-header library.

dropbox/hqq

Official implementation of Half-Quadratic Quantization (HQQ)

opendilab/LightRFT

LightRFT: Light, Efficient, Omni-modal & Reward-model Driven Reinforcement Fine-Tuning Framework

aws-samples/sample-genai-on-eks-starter-kit

A comprehensive toolkit for deploying production-ready Generative AI infrastructure on Amazon EKS. Includes pre-configured components for: 🚀 AI Gateway (LiteLLM) 🤖 LLM Serving (vLLM, SGLang, Ollama)

tairov/llama2.mojo

Inference Llama 2 in one file of pure 🔥

chrisliu298/awesome-llm-unlearning

A resource repository for machine unlearning in large language models

sinanuozdemir/quick-start-guide-to-llms

The Official Repo for "Quick Start Guide to Large Language Models"

dvgodoy/FineTuningLLMs

Official repository of my book "A Hands-On Guide to Fine-Tuning LLMs with PyTorch and Hugging Face"

YangLing0818/buffer-of-thought-llm

[NeurIPS 2024 Spotlight] Buffer of Thoughts: Thought-Augmented Reasoning with Large Language Models

llamastash/llamastash

Zero-overhead, terminal-native local-LLM orchestrator. Run any engine, any model via the same proxy.

NetX-lab/Frontier

Frontier: A Discrete-Event Simulator for Modern LLM Serving

datawhalechina/base-llm

从 NLP 到 LLM 的算法全栈教程，在线阅读地址：https://datawhalechina.github.io/base-llm/

ZJU-LLMs/Agent-Kernel

A MicroKernel Multi-Agents System Framework for Adaptive Social Simulation Powered by LLMs

Tylogi/TyloQuant

Get more intelligence from every bit. Better quantization formats and smarter calibration let larger, stronger models run smoothly on the hardware you already own.

zhanshijinwat/Steel-LLM

Train a 1B LLM with 1T tokens from scratch by personal

AarambhDevHub/aarambh-studio

🦀 Decoder-only LLM built from scratch in pure Rust using Candle — no Python, no PyTorch. Gated DeltaNet + sparse attention, fine-grained MoE, native video/document understanding, long-horizon tool age

a-agmon/rs-graph-llm

High-performance framework for building interactive workflow systems in Rust. Designed for complex workflows and multi-agent systems

mit-han-lab/haq

[CVPR 2019, Oral] HAQ: Hardware-Aware Automated Quantization with Mixed Precision

houtini-ai/houtini-lm

MCP server that saves Claude Code tokens by delegating bounded tasks to local or cloud LLMs. Works with LM Studio, Ollama, vLLM, DeepSeek, Groq, Cerebras.

SqueezeAILab/SqueezeLLM

[ICML 2024] SqueezeLLM: Dense-and-Sparse Quantization

snexus/llm-search

Querying local documents, powered by LLM

Sandermage/sndr_core_engine

SNDR Core Engine (Genesis) — vLLM runtime patch-overlay for Qwen3.6 + Gemma4 on consumer NVIDIA (Ampere sm_86, 2× A5000/3090). Qwen3.6-35B-A3B FP8 ~240 tok/s, 27B-int4 hybrid GDN+Mamba, Gemma4 26B/31B

runebookai/tome

a magical LLM desktop client that makes it easy for *anyone* to use LLMs and MCP

Kenza-AI/sagify

LLMs and Machine Learning done easily

DerryHub/BEVFormer_tensorrt

BEVFormer inference on TensorRT, including INT8 Quantization and Custom TensorRT Plugins (float/half/half2/int8).

SforAiDl/KD_Lib

A Pytorch Knowledge Distillation library for benchmarking and extending works in the domains of Knowledge Distillation, Pruning, and Quantization.

Tebmer/Awesome-Knowledge-Distillation-of-LLMs

This repository collects papers for "A Survey on Knowledge Distillation of Large Language Models". We break down KD into Knowledge Elicitation and Distillation Algorithms, and explore the Skill & Vert

Zhen-Dong/Awesome-Quantization-Papers

List of papers related to neural network quantization in recent AI conferences and journals.

Picovoice/picollm

On-device LLM Inference Powered by X-Bit Quantization

yingpengma/Awesome-Story-Generation

This repository collects an extensive list of awesome papers about Story Generation / Storytelling, exclusively focusing on the era of Large Language Models (LLMs).

Zhen-Dong/HAWQ

Quantization library for PyTorch. Support low-precision and mixed-precision quantization, with hardware implementation through TVM.

baker27727/ds4-8gb-cpu

DeepSeek V4 Flash CPU/NVMe research fork: 78.62 GiB GGUF validated on 7.7 GiB RAM, CPU-only, using demand paging.

swellweb/reame

CPU-first LLM inference server on llama.cpp. Runs useful models on free-tier ARM boxes; rewriting the input made it ~6x faster and more accurate than tuning the engine. MIT, benchmarks and failures in

resume-llm/resume-ai

A local, privacy-first résumé builder using LLMs and Markdown to generate ATS-ready DOCX files with Pandoc — no cloud, no tracking.

openshift-psap/auto-tuning-vllm

Auto-tuning for vllm. Getting the best performance out of your LLM deployment (vllm+guidellm+optuna)

guan-yuan/Awesome-AutoML-and-Lightweight-Models

A list of high-quality (newest) AutoML works and lightweight models including 1.) Neural Architecture Search, 2.) Lightweight Structures, 3.) Model Compression, Quantization and Acceleration, 4.) Hype

Sudharsanselvaraj/Token-Print

Interactive 3D visualization platform for exploring transformer architectures, tensors, and real-time LLM inference.

AMAP-ML/Tree-GRPO

[ICLR 2026] Tree Search for LLM Agent Reinforcement Learning

mgonzs13/llama_ros

llama.cpp (GGUF LLMs) and llava.cpp (GGUF VLMs) for ROS 2

kennethleungty/Finance-LLMs

Comprehensive Compilation of Real-World LLM & AI Agent Use Cases in Financial Services

Gloriaameng/Awesome-Agent-Harness

Agent Harness for Large Language Model Agents: A Survey. Survey on LLM agentharnessengineering with a taxonomy. 110+papers, 23 systems analyzed.

sinanuozdemir/oreilly-hands-on-gpt-llm

Mastering the Art of Scalable and Efficient AI Model Deployment

bd4sur/Nano

电子鹦鹉 / Toy Language Model

huybery/Awesome-Code-LLM

👨‍💻 An awesome and curated list of best code-LLM for research.

wladimiravila/esp32s3-distributed-ai

Distributed 56M-parameter LLM inference across 3 ESP32-S3 boards via ESP-NOW , Split-PLE + KV cache, fully offline.

begin0808/LiveCaption

即時影音雙語字幕翻譯系統。本機離線 AI 語音辨識與翻譯，支援 SenseVoice、Ollama 與 DeepSeek。 | Real-time tab audio bilingual subtitle translation system using local ASR (SenseVoice) and LLMs (Ollama / DeepSeek). 100% offline & pri

SqueezeAILab/KVQuant

[NeurIPS 2024] KVQuant: Towards 10 Million Context Length LLM Inference with KV Cache Quantization

hqhq1025/ai-course-notes

303 份 AI/LLM 中文讲义，支持在线阅读、PDF 下载和 LaTeX 源码查看 | Stanford CS336/CS224R/CS25 | Berkeley LLM Agents | Agent 工程实践

tattn/LocalLLMClient

Swift package to run local LLMs on iOS, macOS, Linux

xinhuangcs/agentmaker

A general-purpose Python framework for building LLM agents and multi-agent systems. "Four lines of code, an agent with memory."

jy-yuan/KIVI

[ICML 2024] KIVI: A Tuning-Free Asymmetric 2bit Quantization for KV Cache

TilelliLab/atome-lm

A ternary, zero-heap tiny language model that runs inside a $2 microcontroller — bit-exact Python <-> C99 <-> Cortex-M3 (QEMU) parity. Apache-2.0.

aimclub/CoScientist

Framework for scientific llm-based multi-agent systems

asukaminato0721/telegram-summary-bot

Summarize group chat with AI, LLM && query group chat, FREE to deploy your own, support img, link meta info, reply to, auto fold result, 支持中文检索.

zengxiao-he/tessera

From teacher to tiles — a from-scratch LLM distillation & serving engine: custom Triton/CUDA kernels, FSDP distillation, paged-KV continuous batching, speculative decoding, a Rust gateway, a JAX oracl

stoyan-stoyanov/llmflows

LLMFlows - Simple, Explicit and Transparent LLM Apps

THUDM/LongCite

LongCite: Enabling LLMs to Generate Fine-grained Citations in Long-context QA

PaddlePaddle/ERNIE-SDK

ERNIE Bot Agent is a Large Language Model (LLM) Agent Framework, powered by the advanced capabilities of ERNIE Bot and the platform resources of Baidu AI Studio.

SynapseKit/SynapseKit

Minimal, async-first Python framework for production LLM apps- 2 hard deps, no magic, no SaaS.

Curated-Awesome-Lists/awesome-llms-fine-tuning

Explore a comprehensive collection of resources, tutorials, papers, tools, and best practices for fine-tuning Large Language Models (LLMs). Perfect for ML practitioners and researchers!

pchunduri6/rag-demystified

An LLM-powered advanced RAG pipeline built from scratch

john-rocky/CoreML-LLM

Run LLMs on Apple devices with CoreML, optimized for Apple Neural Engine + GPU

tpoisonooo/llama.onnx

LLaMa/RWKV onnx models, quantization and testcase

timothystewart6/vllm-gb10

Production ready, bleeding edge vLLM Docker image for the NVIDIA DGX Spark (GB10 / sm_121a).

Leeroo-AI/mergoo

A library for easily merging multiple LLM experts, and efficiently train the merged LLM.

KatherLab/LLMAIx

Document Information Extraction & Anonymization using local LLMs

megvii-research/FQ-ViT

[IJCAI 2022] FQ-ViT: Post-Training Quantization for Fully Quantized Vision Transformer

fypklanfanqie/local-llm-chat-Android-app-with-TTS

A local conversation app for Android that comes with 50 built-in chat characters and supports custom character imports. It imposes no prompt restrictions when using local large language models, and ca

datacrystals/AIStoryWriter

LLM story writer with a focus on high-quality long output based on a user provided prompt.

deepset-ai/haystack-home

Website for Haystack, the open source LLM framework

jia-gao/leanctx

Drop-in prompt compression for production LLM apps. Cut your token bill 40-60% without changing your code. Python SDK, LLMLingua-2, MIT.

dylanhogg/llmgraph

Create knowledge graphs with LLMs

therezor/cardputer-ai

Fully offline tiny LLM chatbot on the ESP32-S3 Cardputer ADV - 8M params, Q4 weights baked into the firmware, ~5 tok/s.

prayangshuuu/hummingbird

hummingbird is a lightweight, zero dependency runtime for massive open source Mixture of Experts (MoE) language models. It unifies SSD, RAM, and VRAM into a single intelligent memory hierarchy, enabli

raphaelmansuy/edgecrab

EdgeCrab 🦀 A Super Powerful Personal Assistant inspired by NousHermes and OpenClaw — Rust-native, blazing-fast terminal UI, ReAct tool loop, multi-provider LLM support, ACP protocol, gateway adapters,

ddzipp/AutoAudit

AutoAudit—— the LLM for Cyber Security 网络安全大语言模型

ahnafnafee/local-llm-pdf-ocr

Convert scanned PDFs and images into searchable documents with any OpenAI-compatible vision server. Run locally with a Web UI or CLI.

KochC/opencode-llm-proxy

Local OpenCode-backed LLM gateway for OpenAI, Anthropic, Gemini, and Responses API-compatible tools, with streaming and tool/function calling.

rwilliamspbg-ops/Ghostlink

Distributed LLM inference fabric for heterogeneous local clusters. Features zero-copy SPSC ring buffers, automated tuning, and an OpenAI-compatible API.

datawhalechina/llm-deploy

大模型/LLM推理和部署理论与实践

RyanLiu112/compute-optimal-tts

Official codebase for "Can 1B LLM Surpass 405B LLM? Rethinking Compute-Optimal Test-Time Scaling".

omar-os/omar

LLM agents are unpredictable. Their coordination doesn't have to be. OMAR delivers deterministic, formally specified orchestration for multi-agent systems.

jieyilong/tree-of-thought-puzzle-solver

The Tree of Thoughts (ToT) framework for solving complex reasoning tasks using LLMs

wpydcr/LLM-Kit

🚀WebUI integrated platform for latest LLMs | 各大语言模型的全流程工具 WebUI 整合包。支持主流大模型API接口和开源模型。支持知识库，数据库，角色扮演，mj文生图，LoRA和全参数微调，数据集制作，live2d等全流程应用工具

cedrickchee/awesome-ml-model-compression

Awesome machine learning model compression research papers, quantization, tools, and learning material.

Kurama622/llm.nvim

A large language model (LLM) plugin for Neovim, provides commands to interact with LLM (like ChatGPT, Copilot, ChatGLM, kimi, deepseek, openrouter and local llms). Support Github models.

JIA-Lab-research/LLMGA

This project is the official implementation of 'LLMGA: Multimodal Large Language Model based Generation Assistant', ECCV2024 Oral

genieincodebottle/aiml-companion

AI-ML Companion: an interactive platform to learn AI & ML by watching it work - 22 tracks, 300+ modules, live visualizations - plus 10 end-to-end portfolio projects from classical ML to production LLM

kanchengw/cnllm

Python toolkit for Chinese LLMs, with flexible batch capacity, structured real-time visulization and automated accumulation for streaming, and explicit feedback on vendor-native parameter validation.

ohdearquant/lattice

Run AI models on Apple Silicon and Raspberry Pi

ServerKarma/ObsidianPrivateAI

Effortlessly chat with your Obsidian notes using a privacy first LLM. Private by design, your notes never leave the device and use local processing only.

artalis-io/bitnet.c

Minimal, zero-dependency LLM inference in pure C11. CPU-first with NEON/AVX2 SIMD. Flash MoE (pread + LRU expert cache). TurboQuant 3-bit KV compression (8.9x less memory per session). 20+ GGUF quant

shouzhuoshouzhuo/FloatTrip

A travel planning Agent powered by LangGraph, Amap POI, route clustering and LLM-based itinerary generation.

HJCheng0602/nanoPD

A from-scratch Prefill/Decode disaggregation inference engine for LLMs

AutoLLM/AutoAgents

Complex question answering in LLMs with enhanced reasoning and information-seeking capabilities.

bestdeejay-design/awesome-ai-handbook

A practical guide to AI: from running your first local model to building your own agents. 52 files covering LLMs, Ollama, RAG, prompt engineering, machine learning, AI agents, and local AI tools.

ZacharyZcR/SecGPT

A Test Project for a Network Security-oriented LLM Tool Emulating AutoGPT

Joyce94/LLM-RLHF-Tuning

LLM Tuning with PEFT (SFT+RM+PPO+DPO with LoRA)

eullm/eullm

Open-source platform for creating, distributing and running sovereign EU-compliant LLMs. Verticalize any model for your domain, language and brand. AI Act ready.

zhijiewong/openharness

Open source local terminal cli with any LLM

RealAlexandreAI/json-repair

🔧 Repair JSON！Solution for JSON Anomalies from LLMs.

Lichang-Chen/InstructZero

Official Implementation of InstructZero; the first framework to optimize bad prompts of ChatGPT(API LLMs) and finally obtain good prompts!

helasaoudi/llm-inspector

The htop for LLM inference see exactly where every GB of VRAM goes and get measured quantization savings.

Drobiazkin/ai-agent-architecture

Build LLM systems you actually control. A free, open engineering book + course — from tokenization to serving your own models. Mechanisms, trade-offs, and numbers, not prompt tips

ausboss/Local-LLM-Langchain

Load local LLMs effortlessly in a Jupyter notebook for testing purposes alongside Langchain or other agents. Contains Oobagooga and KoboldAI versions of the langchain notebooks with examples.

secondpathstudio/privatescribe

100% private AI transcription using local LLMs, fully encrypted database, and extensive administrative tools

sergezuber/FABULA-LLM-5

A local-first coding agent that doesn't take "done" on faith: the engine keeps the run going until the project's own tests pass, and every green run leaves a receipt anyone can replay. Built for one s

manimohans/obsidian-local-llm-helper

An Obsidian plugin to process text, chat with AI, and semantically search your notes — works with any OpenAI-compatible LLM server (Ollama, LM Studio, vLLM, and more).

ShelbyJenkins/llm_client

The Easiest Rust Interface for Local LLMs and an Interface for Deterministic Signals from Probabilistic LLM Vibes

ADT109119/llamacpp-distributed-inference

一個基於 llama.cpp 的分佈式 LLM 推理程式，讓您能夠利用區域網路內的多台電腦協同進行大型語言模型的分佈式推理，使用 Electron 的製作跨平台桌面應用程式操作 UI。

VinRobotics/model-quantization-recipes

Practical quantization recipes for large language models and speech models, from model preparation through deployment-oriented validation.

maximecb/bebelm

CPU-only, pure-Rust implementation of LiquidAI's LFM2.5-8B-A1B LLM

jimdawdy-hub/Local-LLM-Launcher-GUI

A friendly GUI for running local LLMs with vLLM and llama.cpp — hardware-aware traffic-light guidance for every setting. Built on ideas from vllm-cli (Chen-zexi).

Wang-ML-Lab/llm-continual-learning-survey

[CSUR 2025] Continual Learning of Large Language Models: A Comprehensive Survey

linny006/llm-agents-radar

Live-updating index of LLM agent frameworks shipping on GitHub, refreshed every 15 minutes

tanaos/artifex

Small Language Model Inference, Fine-Tuning and Observability.

notwitcheer/llm-bench-rig

Dual-engine (llama.cpp + vLLM) LLM benchmarking pipeline for GGUF & safetensors on NVIDIA GPUs — speed, quality, live dashboard, publishable cards.

AtomFlow-AI/MoleCode

Molecode presents molecules as code and enables LLMs to operate and reason on chemistry directly.

emirsahin1/llm-axe

A simple, intuitive toolkit for quickly implementing LLM powered applications.

alexpospekhov/searchstack-aeo

Searchstack — Open-source AEO/GEO/SEO tech stack for AI-native founders. Monitor your visibility across Google, AI Overviews, ChatGPT, Perplexity, Claude & Grok from one CLI. 22 commands, 9 APIs, llms

unixsysdev/llama-turboquant

TurboQuant 3-bit KV-cache quantization for llama.cpp

AaltoSciComp/llm-examples

This repository contains some LLM deployment examples

DINKIssTyle/DINKIssTyle-Local-LLM-Gateway-with-TTS

A gateway server that integrates local LLMs with the high-quality TTS engine Supertonic, providing a unified interface.

raiyanyahya/llmaker

Selfhost modern LLM stacks. Run the whole fleet from your terminal

genlab-1c/prism

Открытый бенчмарк LLM: какая нейросеть лучше пишет код 1С:Предприятие (BSL). Объективная оценка LLM по методике SMOP с реальным исполнением в 1С — Claude, GPT, Gemini, DeepSeek, YandexGPT, GigaChat.

kr4ckhe4d/local-llm-benchmarks

mohammad-majoony/GraphRAG-Studio

Full-stack GraphRAG application with Django backend and React frontend. Build knowledge graphs, detect communities, and chat with documents using hybrid retrieval and LLM-powered reasoning.

grasp-technologies/grasp-agents

Modular framework for building LLM workflows and multi-agent systems

gmickel/CodeWhisper

CodeWhisper: AI-Powered End-to-End Task Implementation & blazingly fast Codebase-to-LLM Context Bridge

ruankie/ecrivai

Fully automated AI blog writer that uses LangChain and GPT type LLMs for topic selection and content generation

openfabric-systems/simllm

Network-faithful simulation of LLM serving and training deployments

kevindegila/construire-un-llm-de-zero

Construire un LLM de zéro, à la main et en français : les notebooks des 22 chapitres, des tenseurs jusqu'aux agents IA. Le livre et ses visualisations interactives : llmdezero.com

serp-ai/LLaMA-8bit-LoRA

Repository for Chat LLaMA - training a LoRA for the LLaMA (1 or 2) models on HuggingFace with 8-bit or 4-bit quantization. Research only.

MeshInfra/WaferLLM

WaferLLM: Large Language Model Inference at Wafer Scale

ue-patcher/Code_Stitcher

AI Code Stitcher - Paste AI Generated Code - It Just Works! Copy output from Claude, Cursor, ChatGPT, Aider or any other LLM and AI Code Stitcher intelligently applies it to the correct locations in y

shifulegend/project-zero

CPU-optimized LLM inference engine (C)

QingGo/engram-peft

🚀 Engram-PEFT: An unofficial implementation of DeepSeek Engram. Inject high-capacity conditional memory into LLMs via sparse retrieval PEFT without increasing inference FLOPs / DeepSeek Engram 架构的非官方实

lrozewicz/vLLM-Moet-GB10

Run DeepSeek-V4-Flash 304B on a single NVIDIA GB10 / DGX Spark - 2-bit MoE planes, FP4 quality recovery, speculative decoding

caudena/beam_weaver

Elixir-native LangChain, LangGraph, and DeepAgents for traceable LLM apps: OTP workflows, tools, memory, human-in-the-loop, streaming, custom clients/adapters, minimal deps, and WeaveScope tracing.

MaincodeHQ/from-tin-to-tokens

A free textbook on building an LLM inference stack from scratch in Rust.

acupof-ai/arle

The local inference server for coding agents. Pure Rust, one binary, Apple Silicon + NVIDIA. Anthropic + OpenAI APIs; the KV cache survives across turns, so turn 20 starts as fast as turn 2. OPD train

JuneYaooo/awesome-medical-ai

A curated list of open-source medical and healthcare AI projects - LLMs, imaging, multi-agent systems, clinical software

bilalonur/awesome-llm-os

A curated list of awesome resources, tools, research papers, and projects related to the concept of Large Language Model Operating Systems (LLM-OS).

h9-tec/llmviz

Publication-quality LLM architecture figures from config.json — Raschka-style towers, diffs, posters, VRAM math. No weights needed.

bahree/ModelAdaptationBook

Companion code for the Manning book 'LLM Customization and Fine-Tuning.' Adapt open-weights LLMs end to end: prompting and RAG, LoRA/QLoRA, full SFT, distillation, and DPO/RLHF alignment.

dipampaul17/KVSplit

Run larger LLMs with longer contexts on Apple Silicon by using differentiated precision for KV cache quantization. KVSplit enables 8-bit keys & 4-bit values, reducing memory by 59% with <1% quality lo

upmem/upmem_llm_framework

UPMEM LLM Framework allows profiling PyTorch layers and functions and simulate those layers/functions with a given hardware profile.

Aliu-AiRobot/ESEILANE

High-performance Knowledge Graph engine for AI, LLMs, and GraphRAG — built for the next generation of intelligent applications.

openmake/openmake_llm

Open-source, self-hosted AI workspace for local and open-weight LLMs with vLLM, LiteLLM, autonomous agents, MCP tools, deep research, artifacts, and BYOK.

ahb-sjsu/turboquant-pro

Consumer-aware compression for embedding indexes and LLM KV caches — compress by the metric the downstream consumer actually uses. PCA-Matryoshka + TurboQuant (27x @ 99.8% recall@10), asymmetric K/V,

zz-haooo/Meta-Team

The implementation of the paper "Evolve as a Team: Collaborative Self-Evolution for LLM-based Multi-Agent Systems"

deveix/react-native-apple-llm

React Native Apple LLM plugin using Foundation Models

Sreyan88/GAMA

Code for the paper: GAMA: A Large Audio-Language Model with Advanced Audio Understanding and Complex Reasoning Abilities

tolitius/cupel

discover LLMs punching above their weight

OnlyTerp/turboquant

First open-source implementation of Google TurboQuant (ICLR 2026) -- near-optimal KV cache compression for LLM inference. 5x compression with near-zero quality loss.

PKU-Alignment/beavertails

BeaverTails is a collection of datasets designed to facilitate research on safety alignment in large language models (LLMs).

EmilResearch/RAGless

RAGless is a semantic retrieval system that answers questions about your documentation, without using an LLM at runtime.

pccxai/pccx-FPGA-NPU-LLM-kv260

KV260 integration lane for PCCX™ v002 LLM IP-core bring-up, validation, and board/runtime evidence.

Simpl3x3/ESEILANE

High-performance Knowledge Graph engine for AI, LLMs, and GraphRAG — built for the next generation of intelligent applications.

esanmohammad/Anvil

Provider-agnostic AI dev pipeline: clarify → plan → build → review → PR across your repos, mixing LLM providers per stage with your own keys. No vendor lock-in, no markup.

pur4v/p2ptokens

Distributed, peer-to-peer, decentralized network for LLM inference — peers seed & leech completions, metered by BitTorrent-protocol-style upload/download ratios. OpenAI-compatible · Rust · libp2p.

AspadaX/tiny-llm

A minimal MiniCPM5-1B model inference engine built purely in Rust

Aesthisia/LLMinator

Gradio based tool to run opensource LLM models directly from Huggingface

UnlikedOne/diffuse

Distributed peer-to-peer LLM inference. Your prompt never leaves your device in clear text.

IST-DASLab/GSQ

Gumbel-Softmax post-training quantization for LLMs (1–3 bit scalar, INT/GGUF-compatible).

pillowsofwind/Knowledge-Conflicts-Survey

[EMNLP 2024] The official GitHub repo for the survey paper "Knowledge Conflicts for LLMs: A Survey"

jiahongsigma/Efficient-LLM-Inference-Serving-Systems

Why is LLM inference slow — and how do you make it fast? A hands-on, first-principles course: roofline → KV cache → quantization → parallelism → vLLM/SGLang, with GPU labs on open models.

SamsungLabs/NanoQuant

[ICML 2026] NanoQuant: Efficient Sub-1-Bit Quantization of Large Language Models

EngramMemory/engram-memory

The highest-scoring AI memory system ever benchmarked that isn't reliant on LLM reranking. And it's free & burns less tokens.

spranab/contextcache

Tool schemas get re-prefilled on every LLM request. ContextCache compiles them into a KV cache once, stores it on disk and reuses it across users and sessions — TTFT stays near 200ms with 50 tools ins

matteo-grella/fucina

CPU-first, eager-execution tensor/autograd runtime and LLM inference engine written in pure Zig

atharva557/Prompt-Chaining

Chain two LLMs — local or cloud: a Prompter that refines your idea, and a Coder that writes the code.

stfurkan/bitgpu

Fast WebGPU runtime for 1-bit (binary-weight) LLMs in the browser. Bit-exact, zero runtime dependencies.

Lzxpan/CodeWorker

離線 Windows 本地 LLM 程式碼助理，支援 Gemma4、Qwen 3.5 影音、圖文分析，可分析整個專案或釘選檔案針對單檔分析，隱私優先的本機專案補助LLM。

undreamai/LlamaLib

Cross-Platform High-Level LLM Library

RightNow-AI/TIDE

Dynamic per-token early exit for LLM inference. Skip layers tokens don't need

apanariello4/merge-and-rebase

Model merging, task-vector rebasin, and fine-tuning for vision and LLM models.

ariannamethod/nanollama

Train Llama 3 models from scratch. Any scale, any personality. By Arianna Method.

AbdelStark/turboquant

Rust implementation of Google's TurboQuant algorithm for vector quantization

MaximeDeconinck/paddock

Which LLMs fit your Apple Silicon Mac, how fast they'll run, and how to launch them, in one command.

Sjay-Wang/K-ragrec

This is the pytorch implementation for our ACL paper Knowledge Graph Retrieval-Augmented Generation for LLM-based Recommendation.

jndiogo/sibila

Extract structured data from local or remote LLM models

ankitvirdi4/awesome-llm-cost

Tools, libraries, papers, and patterns for reducing the cost of running large language models in production.

Rahul-14507/MELLM

Lightweight Modular AI Routing Engine for Local LLMs — Run specialised experts efficiently on consumer GPUs using smart Mixture-of-Experts routing.

LSXPrime/ProseFlow

Your universal AI text processor, powered by local and cloud LLMs. Edit, refactor, and transform text in any application on Windows, macOS, and Linux.

zb614433612/CodeCraft

基于 DeepSeek 大语言模型的 AI 编程助手 Agent —— 自然语言驱动的终端编码助手。支持自动读写文件、代码搜索、命令执行、Git 管理，复杂任务智能拆解与多 Agent 并行执行。

JINO-ROHIT/tachyon

a LLM inference engine to run on consumer hardware

Alberto-Codes/turboquant-vllm

TurboQuant KV cache compression plugin for vLLM — asymmetric K/V, 8 models validated, consumer GPUs

isartor-ai/Isartor

Open-source Prompt Firewall — deflect up to 95% of redundant LLM traffic before it leaves your infrastructure. Documents: https://isartor-ai.github.io/Isartor/index.html

andersondanieln/hexllama

A beautifully crafted desktop client for running and managing local LLMs via llama.cpp.

back2matching/turboquant

First open-source TurboQuant KV cache compression for LLM inference. Drop-in for HuggingFace. pip install turboquant.

taylorsatula/TeaLeaves

End-to-end pipeline for seeing how LLMs actually process your prompts. Capture attention across every layer, render heatmaps and cooking curves, compare variants with evidence — not vibes.

Snseam/awesome-agent-memory

Curated reading list & living survey on long-term memory for LLM agents — 989 papers, 529 archived PDFs, deep notes mapped to memory-kernel modules. Feeds the Ymem Research Radar.

Venkat2811/wombatkv

Object-storage-native KV cache for LLM inference & RL. Cross-restart, cross-conversation, cross-engine via shared S3 bucket.

ContextFit/cf

Token-native agent memory retrieval for LLMs, without embedding APIs or vector databases.

xigh/herbert-rs

Local LLM inference engine written from scratch in Rust — hand-written AVX-512 assembly kernels, Metal & Vulkan compute shaders. Supports Qwen3, Mistral3, ... Q4/INT8/BF16 quantization.

kujirahand/book-local-llm-sample

ローカルLLMの解説本のサンプル一式

FAI-Solutions/open-webui-extensions

A growing collection of practical filter, pipeline & tool extensions for Open WebUI, solving real usability gaps in local & cloud LLM workflows.

jcnade/COBOL-AI-LLM-Framework

enescingoz/mac-llm-bench

Community benchmark database for running LLMs on Apple Silicon Macs

TaewoooPark/DRIFT

🧠 One LLM, split across a Mac (Apple MPS) + a Windows PC (NVIDIA CUDA) — heterogeneous pipeline inference over a framework-neutral wire, bit-for-bit identical to single-machine. No torch.distributed,

mobilint/vllm-mblt

An out-of-tree vLLM plugin for Mobilint NPU runtime integration.

SparkEngineAI/QuantClaw-plugin

QuantClaw is a plug-and-play task-type routing quantization plugin for OpenClaw.

SheikhSarvar/pyrag-core

Open-source, Python-first RAG platform with FastAPI APIs, LangGraph-native agents, pluggable vector stores (Qdrant/Weaviate/Milvus/pgvector/Elasticsearch), and multi-provider LLM support (OpenAI/Anthr

videlalvaro/llm-arithmetic-internals

Mechanistic experiments on how LLMs represent and compute arithmetic internally, with strict no-parser controls, reproducible audits, and an interactive article.

minskylab/auto-rust

auto-rust is an experimental project that automatically generate Rust code with LLM (Large Language Models) during compilation, utilizing procedural macros.

danilodevhub/turboquant-js

TypeScript implementation of Google's TurboQuant algorithm for near-optimal vector quantization. Zero dependencies, works in Node.js and browsers.

AbdelrhmanEbied/Tiny-MoE

Tiny-MoE is a lightweight Mixture-of-Experts language model built entirely from scratch in native PyTorch and trained end-to-end on Kaggle using free 2× NVIDIA T4 GPUs. The project implements modern L

radcrew/radcrew-unveiled

RadCrew.org website with LLM-powered chatbot

embedl/flash-head

FlashHead: Efficient Drop-In Replacement for the Classification Head in Language Model Inference

cogflows/promptcode-vscode

PromptCode: The VS Code extension that saves the day when your AI code agent (Cursor, etc.) hits a wall. You're in control—select files, craft precise prompts, and effortlessly apply fixes using your

golab-ai/Suiren-Property-Prediction

A PyTorch-based fine-tuning framework for adapting Suiren pre-trained molecular models to various downstream property prediction tasks.

InfraWhisperer/llmtop

htop for your LLM inference cluster

ionic-team/capacitor-local-llm

A Capacitor plugin for running on-device LLMs, with support for text generation, image generation, and session management on iOS and Android.

Cross2pro/DeepSeek-OCR-Dashboard

An out-of-the-box local Web UI for DeepSeek-OCR. Built with FastAPI + Vue.js, it supports PDF/Image uploads, progress tracking, and result visualization with bounding boxes. Easily experience the p

lna-lab/blackwell-geforce-nvfp4-gemm

NVFP4 inference on Blackwell GeForce (RTX 5090/5080/5070 Ti/RTX PRO 6000) — SM120 patches for vLLM + FlashInfer + CUTLASS. 175 tok/s on Qwen3.6-35B MoE.

sakana-ctf/v_llama_cpp

lightweight V wrapper for Llama.cpp, enabling efficient LLM execution.

heldernoid/openstitch

Draw cool web UIs with local LLMs.

HRI-EU/AttentiveSupport

llm-based robot that intervenes only when needed

parvbhullar/superpilot

LLMs based multi-model framework for building AI apps.

dhcgn/paperless-ngx-privatemode-ai

Help with documents in Paperless NGX with confidential LLM by Privatemode.ai

actypedef/SharQ

Code for the paper “SharQ: Bridging Activation Sparsity and FP4 Quantization for LLM Inference”

Alex188dot/ziggy-llm

Zig GGUF inference engine for Apple Silicon

Firmamento-Technologies/TurboQuant

Near-optimal vector quantization from Google's ICLR 2026 paper — 95% recall, 5x compression, zero preprocessing, pure Python FAISS replacement

FastLM/CSV-Decode

CSV-Decode: Certifiable Sub-Vocabulary Decoding for Efficient Large Language Model Inference

di37/finetuning-quantize-evaluate

Fine-Tune, Quantize, Evaluate: The Complete Guide — LLMs, VLMs, and Embedding Models

maeddesg/vulkanforge

LLM inference engine for AMD RDNA4 — Rust + Vulkan compute shaders, gguf & native FP8.

devnen/vllm-windows

Patched native-Windows build of vLLM. Three Windows-specific fixes (CPU-relay for Gloo, Qwen3 reasoning parser, wildcard model name) on top of SystemPanic 0.19.0. Engine for devnen/qwen3.6-windows-ser

VectorPeak/LLM-Wiki

A structured knowledge base for large language model technologies, covering LLMs, Agents, RAG, model training, evaluation methodologies, and AI engineering practices.

mikecvet/nl-sh

The Natural Language Shell integrates OpenAI's GPTs, Anthropic's Claude, or local GGUF-formatted LLMs directly into the terminal experience, allowing operators to describe their tasks in either POSIX

SuleynanAuir/Enterprise-Enhanced-LLaMA-Factory-Pro-Advanced-FineTuning-Local-Deployment-Pipeline

LLaMA Factory is an end-to-end LLM fine-tuning and deployment pipeline, integrating data augmentation & engineering, cloud-based training, and cloud data management. With one-click fine-tuning and loc

MerkyorLynn/lynn-engine

Lynn 原生 LLM 推理引擎 · W4A8/NVFP4 量化 · 自写 CUDA/Triton kernel · MoE · 投机解码 | Lynn-native LLM inference engine for NVIDIA Blackwell

ReyChiaro/diffusers-tuner

A Lightweight, Configuration-Driven, Flexible Fine-Tuning Framework for 🤗 Diffusers

lna-lab/gemma4-12b-vllm-sm120

Reproducible recipe: serve abliterated Gemma-4-12B (gemma4_unified) at 50-118 tok/s on no-NVLink Blackwell (SM120) via vLLM nightly + ModelOpt FP8/NVFP4 + MTP spec-decode.

0xzuky/code-llm-completion

AI/ML research project on AMD ROCm

JihoonJeong/Neural-MRI

Model Resonance Imaging — visualize LLM internals like a brain MRI

Jackohhhh/MedLLM-Finetuning

一个开箱即用、用于二分类任务的大语言微调模型框架。An out-of-the-box LLM fine-tuning framework for medical binary classification.

Emmimal/llm-eval-layer

A lightweight LLM evaluation layer that turns subjective model outputs into deterministic decisions by scoring attribution, specificity, relevance, and context quality—designed to detect hallucination

NadavIs56/AutoGPT-Telegram-bot

AutoGPT Telegram Bot is a Python-based chatbot developed for a self-learning project. It leverages the power of OpenAI's GPT language model to answer user questions and maintain conversation history f

qybaihe/bla

🛡️ BLA (Blind LLM Agents) — Adversarial defense shield that protects binaries, Electron apps, JS bundles, Python, and WebAssembly against AI reverse-engineering agents like REA. Poison -> Exhaust -> H

mmparko/dsh-purge

DeepSeek Harness 破甲 (mirror of YuJunZhiXue/dsh-purge) — 上游即将删库，此为本仓库全量镜像：含全部分支/73 个 tag/完整提交历史。原作者署名与二开要求见 README 署名章节。

Minglink/dsh-infinite-gen-5

DeepSeek v4.1 flash 最强逆向工具（无限五代） 求 Star 收藏 ⭐欢迎大家提交项目的改进

Mesh-LLM/mesh-llm

Distributed AI/LLM for the people. Share compute privately or publicly to power your agents and chat.

Fei2411/DeepSeek-Bot

A team of long-lived bots for DeepSeek Harness, with main bots, bot-to-bot messages and group chats.

dsh-tauri/deepseek-harness-desktop

DSH Tauri 桌面版 | Better DSH desktop, zero environment setup, preset plugins, Windows / macOS / Linux.

karthink/gptel

A simple, extensible LLM client for Emacs

xmanrui/dsh-im

通过扫码或机器人凭据把IM机器人接入DeepSeek Harness（支持飞书、微信、钉钉、企业微信、QQ、Slack、Telegram、Discord和WhatsApp）。 Connect IM bots to DeepSeek Harness via QR code or credentials (9 channels).

lynote-ai/humanize-text

Open-source AI humanizer: rewrite AI-generated text into natural, human-sounding prose. LLM rewriting + multi-engine translation, with step-by-step traces. Python CLI, n8n & Docker.

modelflare-ai/AnyModelBot

🤖 Open-source, local-first alternative to Grok Bot, OpenAI dots and Claude Duet. Your own AI bots on any model: GPT, Claude, Gemini, Grok, DeepSeek, Kimi, GLM and 70+ more through one Modelflare key.

DSH-APP/DSHA

免 ROOT 免 Termux，在手机上跑 DeepSeek Harness。完整 Ubuntu 环境 + proroot 零 ptrace 开销 · AI 输出实时上屏 · ADB 直连 · 数据不丢

yoloytang/Awesome-LLMs-for-Video-Understanding

🔥🔥🔥 [IEEE TCSVT] Latest Papers, Codes and Datasets on Vid-LLMs.

yunlong10/Awesome-LLMs-for-Video-Understanding

🔥🔥🔥 [IEEE TCSVT] Latest Papers, Codes and Datasets on Vid-LLMs.

UKGovernmentBEIS/inspect_ai

Inspect: A framework for large language model evaluations

spiceai/spiceai

Add a real-time analytics node to your operational database. Spice is a portable, accelerated SQL query, search, and LLM-inference engine in Rust for data-grounded AI apps and agents.

zjunlp/LLMAgentPapers

Must-read Papers on LLM Agents.

SharpAI/DeepCamera

Open-Source AI Camera Skills Platform, AI NVR & CCTV Surveillance. Local VLM video analysis with Qwen, DeepSeek, SmolVLM, LLaVA, YOLO26. LLM-powered agentic security camera agent — watches, understand

b4rtaz/distributed-llama

Distributed LLM inference. Connect home devices into a powerful cluster to accelerate LLM inference. More devices means faster inference.

MLNLP-World/LLMs-from-scratch-CN

LLMs-from-scratch项目中文翻译

ulab-uiuc/LLMRouter

LLMRouter: An Open-Source Library for LLM Routing

datawhalechina/hugging-llm

HuggingLLM, Hugging Future.

hegelai/prompttools

Open-source tools for prompt testing and experimentation, with support for both LLMs (e.g. OpenAI, LLaMA) and vector databases (e.g. Chroma, Weaviate, LanceDB).

bbruceyuan/Hands-On-Large-Language-Models-CN

中文翻译的 Hands-On-Large-Language-Models (hands-on-llms)，动手学习大模型

Luce-Org/lucebox

LLM speculative inference server for heterogeneous hardware & consumer GPUs

yaojingang/yao-open-prompts

Practical bilingual AI prompt library for prompt engineering, work, learning, content, marketing, and everyday use.

shaobeichen/dsh-pocket

把 DeepSeek Harness 装进你的口袋：电脑上跑 dsh web，手机扫码即同步访问（局域网 + 公网，实时同屏）Put DeepSeek Harness in your pocket: run dsh web on your computer and access it synchronously by scanning a QR code on your phone (LAN +

huggingface/nanotron

Minimalistic large language model 3D-parallelism training

mll-lab-nu/RAGEN

Agent RL framework for LLM agents: multi-turn reinforcement learning with StarPO and reasoning-collapse diagnostics

deepseek-ai/deepseek-recipe

FasterDecoding/Medusa

Medusa: Simple Framework for Accelerating LLM Generation with Multiple Decoding Heads

vllm-project/production-stack

vLLM’s reference system for K8S-native cluster-wide deployment with community-driven performance optimization

AmberSahdev/Open-Interface

Control Any Computer Using LLMs.

syv-ai/HyperQwen

Serve large Qwen models fast on the GPUs you actually own. Qwen3.8-27B on a single 24 GB card with vLLM: 127 tok/s single-user (381 when the answer quotes the prompt), ~1,035 tok/s at 64 concurrent, 1

AnswerDotAI/llms-txt

The /llms.txt file, helping language models use your website

km1994/LLMs_interview_notes

该仓库主要记录 大模型（LLMs） 算法工程师相关的面试题

mirage-project/mirage

Mirage Persistent Kernel: Compiling LLMs into a MegaKernel

QIN2DIM/hcaptcha-challenger

🥂 Gracefully face hCaptcha challenge with multimodal large language model.

NovaSky-AI/SkyRL

SkyRL: A Modular Full-stack RL Library for LLMs

google/tunix

A Lightweight LLM Post-Training Library

eugr/spark-vllm-docker

Docker configuration for running VLLM on dual DGX Sparks

howl-anderson/unlocking-the-power-of-llms

使用 Prompts 和 Chains 让 ChatGPT 成为神奇的生产力工具！Unlocking the power of LLMs.

microsoft/Olive

Olive: Simplify ML Model Finetuning, Conversion, Quantization, and Optimization for CPUs, GPUs and NPUs.

bbruceyuan/LLMs-Zero-to-Hero

从无名小卒到大模型（LLM）大英雄~ 欢迎关注后续！！！

AmberLJC/LLMSys-PaperList

Large Language Model (LLM) Systems Paper List

trypromptly/LLMStack

No-code multi-agent framework to build LLM Agents, workflows and applications with your data

trhhubsd/ARTEX

ARTEX · AI autonomous penetration testing system (multi-agent LLM, Go + Next.js) · AI 自主渗透测试系统 · 百度 agent+ 攻防挑战赛冠军

Small-tailqwq/dsh-deep-whale

Whale Girl skin series for DeepSeek Harness. 适用于 DeepSeek Harness 的，鲸鱼娘系列皮肤。

anthropics/anthropic-sdk-typescript

Access to Anthropic's safety-first language model APIs in TypeScript

dsh-tauri-desk/deepseek-harness-desktop

DeepSeek Harness Tauri 桌面版 | Only 5mb installer, zero environment setup, preset plugins, Windows / macOS / Linux.

ZJU4HealthCare/Foundations-of-Medical-LLMs

Foundations of Medical Large Language Model Learning

datawhalechina/zero-to-sglang

Official SGLang x Datawhale course on LLM inference: understand inference, build a mini-sglang from scratch, then read the real SGLang source and land your first PR. Available in English and Chinese.

AI-in-Health/MedLLMsPracticalGuide

[Nature Reviews Bioengineering🔥] Application of Large Language Models in Medicine. A curated list of practical guide resources of Medical LLMs (Medical LLMs Tree, Tables, and Papers)

patterns-ai-core/langchainrb

Build LLM-powered applications in Ruby

vllm-project/vllm-metal

Community maintained hardware plugin for vLLM on Apple Silicon

NanmiCoder/dsh-agent-teams

AgentTeams plugin for DeepSeek Harness

llmgenai/LLMInterviewQuestions

This repository contains LLM (Large language model) interview question asked in top companies like Google, Nvidia , Meta , Microsoft & fortune 500 companies.

e-p-armstrong/augmentoolkit

Create Custom LLMs

kwaroran/Risuai

Make your own story. User-friendly software for LLM roleplaying

sybil-solutions/local-studio

Control panel for VLLM, Sglang, llama.cpp, exllamav3

MorDavid/BruteForceAI

Advanced LLM-powered brute-force tool combining AI intelligence with automated login attacks

ShadowHackrs/Jailbreaks-GPT-Gemini-deepseek-

Jailbreaks GPT, Sora, Claude, Gemini ,deepseek this prompt unlocks rage mode

DSH-EAC/DSH-Desktop-EAC

Embracing All Creation — A plugin suite for harmonious coexistence of hundreds of plugins / 揽尽万象 —— 致力于让数百个插件和谐共存的插件整合包

0xSero/turboquant

TurboQuant: Near-optimal KV cache quantization for LLM inference (3-bit keys, 2-bit values) with Triton kernels + vLLM integration

turboderp-org/exllamav3

An optimized quantization and inference library for running LLMs locally on modern consumer-class GPUs

Devin-AXIS/deepseek-design

DeepSeek Harness 可编辑设计系统：AI 生成、可视化编辑、模板市场与 PPT｜Native Design & PPT Studio for DeepSeek Harness.

APIParkLab/APIPark

Cloud native, ultra-high performance AI&API gateway, LLM API management, distribution system, open platform, supporting all AI APIs.🦄云原生、超高性能 AI&API网关，LLM API 管理、分发系统、开放平台，支持所有AI API，不限于OpenAI、Azure、A

tmylla/Awesome-LLM4Cybersecurity

An overview of LLMs for cybersecurity.

nlmatics/llmsherpa

Developer APIs to Accelerate LLM Projects

scadastrangelove/awesome-ai-security-tools

A curated list of public-source, research, and commercial tools for AI security and AI-assisted cybersecurity — autotriage, agent security, AI/ML supply chain, pentest agents, AI SAST, LLM-driven fuzz

facebookresearch/coconut

Training Large Language Model to Reason in a Continuous Latent Space

JShollaj/awesome-llm-interpretability

A curated list of Large Language Model (LLM) Interpretability resources.

zouyuxuan122/DSH-Desktop-EAC

DeepSeek Harness Desktop (dsh-desktop). EAC: Embracing All Creation (揽尽万象). Bundled Node.js runtime with full dsh-CLI kernel, one-click startup, 10 built-in UI themes.

youngyangyang04/llm-master

大模型（LLM）全栈学习路线与中文教程🔥：覆盖 Prompt Engineering、RAG、AI Agent、MCP、微调、模型部署、Transformer、AI 编程与大厂面试，从入门到生产实践。

nokia-applied-research/AnyJev

Turn any LLM into a Jev-style decision model: typed decisions, real probabilities, no training. (continue updating, welcome any issue and PR request)

kyutai-labs/unmute

Make text LLMs listen and speak

rasbt/llm-architecture-gallery

LLM Architecture Gallery source data

MiaAI-Lab/DeepSeek-v4-Flash-DSpark-2x-DGX-Spark

DeepSeek-v4-Flash 0731 recipe for 2x DGX Sparks

1CatAI/1Cat-vLLM

V100 / SM70-focused vLLM engineering fork for modern LLM inference.