Semantic Search Engine for Your Own Code

A local-first semantic search engine for your codebase. Search your own code using natural language — without uploading your source code to any external server.

pip install semantic-search
semantic-search index ./my-project
semantic-search query "Where are user permissions checked?"

Your code. Your machine. Your search.

---

What is this?

Developers ask questions about their code in natural language and receive the most relevant files, functions, classes, and code fragments as results:

$ semantic-search query "Where is account deletion permission checked?"

Query:
"Where is account deletion permission checked?"

1. src/auth/permissions.py:5     can_delete_account()      (function, score         0.52)
2. src/users/security.py:18      check_user_permissions()   (method,     score         0.48)
3. src/accounts/service.py:103    delete_account()           (function, score         0.44)

Unlike grep or IDE search, it does not require knowing the exact identifier or file name. It finds code by meaning, not just by matching strings:

"Where are permissions checked before deleting an account?"

can find:

def can_delete_account(user):
    return user.role == "admin"

even though the query and the code share almost no literal words.

How it works

Code
  ↓
Parser (tree-sitter AST)
  ↓
Semantic chunks (functions, classes, methods, …)
  ↓
Local embeddings (ONNX neural model or built-in hashing engine)
  ↓
Local vector database (on your disk)
  ↓
Natural Language Query
  ↓
Semantic Search
  ↓
Relevant Code (with file and line locations)

1. Indexing

semantic-search index ./my-project scans the repository, parses every supported source file into semantic symbols (functions, classes, methods, interfaces, structs, …), splits them into chunks, embeds each chunk into a numerical vector, and stores everything in a local vector database.

2. Searching

semantic-search query "…" embeds your question with the same local model, finds the most similar code chunks, ranks them, and prints the results with exact file:line locations. Pass --show-code to print the matched code itself.

3. Updating

semantic-search update ./my-project re-scans the repo and only re-embeds the files that actually changed (incremental indexing). On large repositories this turns a full re-index (minutes) into a few seconds.

Privacy

Your code stays on your machine.

The project is designed to be local-first by default:

• indexing runs locally
• parsing runs locally
• embeddings are generated locally
• the vector database lives on your disk
• search queries never leave your machine
• no source code, embeddings, file names, or metadata are sent anywhere

The only optional network activity is the first run of a neural embedding model, when the model is downloaded once from Hugging Face into a local cache (then fully offline afterwards). If you prefer zero downloads, use the built-in hashing embedder (provider = "hash") — deterministic, dependency-free, and fully offline.

Quick start — no terminal needed

The repository ships ready-to-run launchers for every operating system. On first launch they create a local Python environment (one-time), install the tool inside it, and open an interactive menu:

Operating system	How to activate
Windows	Double-click activar.bat
macOS	Double-click activar.command
Linux	Run ./activar.sh (or double-click it after chmod +x activar.sh)

The menu stays open until you close it, and lets you index a repository, ask questions, view status, update an index, or see the configuration. You can also type any CLI command directly in the prompt, for example:

index ./my-project
query "where are permissions checked?" --show-code
status

The launchers accept the normal CLI arguments too, for example from a terminal on any OS:

python launch.py index ./my-project
python launch.py query "where are permissions checked?" --show-code

Everything is installed inside a local .venv folder that belongs to the repository (and is ignored by git), so your system Python and global packages are never touched. All three operating systems are supported and tested in CI.

Installation

Requirements: Python 3.10+ (Windows, macOS, Linux).

This project is distributed through its GitHub repository:

pip install git+https://github.com/novrekdevelop/searchingengineforyourowncode.git

or clone it and install locally:

git clone https://github.com/novrekdevelop/searchingengineforyourowncode.git
cd searchingengineforyourowncode
pip install .

Heads-up: a package named semantic-search also exists on PyPI, but it belongs to an unrelated project. Always install from this repository — never from the bare pip install semantic-search command.

For development (editable install + test/lint tooling):

git clone https://github.com/novrekdevelop/searchingengineforyourowncode.git
cd searchingengineforyourowncode
pip install -e ".[dev]"

Installing from source gives you everything: the CLI, the parsers (tree-sitter grammars for 11 languages), and the built-in hashing embedder that works out of the box with zero downloads and zero model files.

Optional: better semantic quality (neural embeddings)

By default the tool uses a fast, deterministic hashing embedder that works fully offline. For noticeably better semantic matches, install a local neural embedding model:

# From a local clone:
pip install ".[embeddings]"

# Or directly from GitHub:
pip install "semantic-search[embeddings] @ git+https://github.com/novrekdevelop/searchingengineforyourowncode.git"

On first use, the model (BAAI/bge-small-en-v1.5) is downloaded once and cached locally. Afterwards everything runs fully offline. Nothing is ever sent to a server: the model runs on your machine, inside ONNX Runtime.

Usage

Index a repository

semantic-search index ./my-project

Progress is printed as files are scanned and embedded. The index is stored locally (see semantic-search config data-dir).

Ask a question

semantic-search query "Where is authentication implemented?"
semantic-search query "How are failed payments handled?"
semantic-search query "Where are API keys validated?" --top-k 5
semantic-search query "Which functions deal with rate limiting?" --show-code

Update after code changes

semantic-search update ./my-project     # only changed files are re-embedded

Status

$ semantic-search status
Repository: C:\Users\you\my-project
Files indexed: 438
Code chunks: 8,421
Languages: Python, TypeScript
Index size: 124 MB

Clear (delete an index)

semantic-search clear

Configuration

semantic-search config init        # writes config.toml with defaults
semantic-search config show        # print the effective configuration
semantic-search config path         # print the user config file path
semantic-search config data-dir

Settings are layered (later wins):

1. built-in defaults
2. user config file (~/.config/semantic-search/config.toml)
3. project config file (<repo>/.semantic-search.toml)
4. SEMANTIC_SEARCH_* environment variables

Example config.toml:

[embedding]
provider = "hash"                    # "hash" | "auto" | "fastembed"

[vector_store]
top_k = 10

[indexing]
chunk_size = 1200

[scanner]
ignore_directories = [".git", "node_modules", "dist", "build", "venv"]
ignore_files = [".env", ".env.*", "*.pem", "*.key", "credentials.*", "secrets.*"]
languages = ["python", "typescript"]
# extensions = {".py" = "python"}

Supported languages

Python · JavaScript · TypeScript (incl. TSX) · Java · Go · Rust · C · C++ · C# · PHP · Ruby — via tree-sitter grammars, with a heuristic fallback when a grammar is missing. Adding a language means adding one grammar dependency, and a small rule table in src/semantic_search/parser/registry.py.

Security: secrets are never indexed

By default, files like the following are skipped automatically:

.env              .env.*         *.pem          *.key
credentials.*    secrets.*       *.min.js       *.map

You can extend these rules via configuration. Ignored files are never read into the index — no source text, no hashes, no embeddings.

Architecture

The application is split into small, replaceable modules:

src/semantic_search/
├── cli/          click command-line interface
├── scanner/      repository walking, ignore rules, language detection
├── parser/       tree-sitter AST parsing, per-language symbol rules
├── chunker/      semantic chunking (functions, classes, …)
├── embeddings/   embedding engines (hash, fastembed/ONNX)
├── vector_store/ flat numpy vector database (+ JSONL metadata
├── search/       query embedding, nearest-neighbor search
├── ranking/      semantic + lexical score fusion
├── index/        orchestration, incremental indexing, index paths
├── models/       shared data models
├── config/       layered TOML configuration
└── util/         small helpers (locks, text tokenization)

Embedding engines implement a 20-line interface (EmbeddingEngine.embed(texts) -> np.ndarray). Swap in the neural model or a future grid of engines without touching anything else. The vector store is a single class too: replace it with FAISS/LanceDB/Qdrant later by implementing the same four methods, no other module changes.

See docs/architecture.md and docs/decisions.md for details and rationale.

Hybrid search

Semantic search finds conceptual matches; lexical (text) search finds exact identifiers (refresh_token, jwt.verify, …). The two complement each other:

semantic-search query "JWT refresh token" --hybrid

hybrid search is enabled per-query with --hybrid, or permanently via search.hybrid = true in config. The ranking fuses both scores (configurable weight, search.semantic_weight).

It is also available as a library:

from semantic_search.index import IndexManager
from semantic_search.config import load_config

manager = IndexManager(config=load_config(".")
results = manager.search("Where are database connections created?", repo_dir=".")
for r in results:
    print(r.rank, r.score, r.chunk.file, r.chunk.symbol)

Tests

pip install -e ".[dev]"
pytest

The suite covers the scanner/ignore matcher, parser, chunker, hashing embedder, vector store, search ranking, config loader, incremental indexing, and the CLI end-to-end (no network, no external services). Tests are hermetic: they never touch your real repositories.

Roadmap

MVP (done)

• CLI (index, query, update, status, clear, config)
• Repository indexing
• Source file detection with ignore rules
• Code parsing (tree-sitter, 11 languages)
• Semantic code chunking
• Local embeddings (hash + optional fastembed/ONNX)
• Local vector database (flat numpy store)
• Semantic search
• Result metadata with file and line locations
• Incremental indexing
• Configuration (layered TOML)
• Tests
• README
• LICENSE
• CONTRIBUTING.md

V2

• Hybrid search (already scaffolded: --hybrid)
• More programming languages
• Language filters
• File filters
• Symbol filters (type:function, path:auth/*)
• Improved ranking (BM25, cross-encoder re-rank)
• Local web interface

V3

• VS Code extension
• Other IDE integrations
• Dependency-aware search
• Similar-code search
• Explanation: local LLM summarizes why each result matched
• Repository-architecture questions (RAG over your codebase)

Why Python?

Python was chosen for the MVP because of its ecosystem: tree-sitter bindings, ONNX-based local embeddings, numpy, and developer tooling are first- class. The architecture keeps each heavy component behind a small interface, so it could be reimplemented in another language later without redesigning the whole system.

License

MIT. See CONTRIBUTING.md to get involved.