Build an AI-Powered Search Engine with LLMs: Complete Tutorial

Traditional keyword search fails when users ask natural language questions. "How do I fix the login bug on mobile?" won't match a document titled "Authentication Flow Troubleshooting — iOS and Android." AI-powered search understands intent, not just keywords. Here's how to build one.

Architecture: AI Search Pipeline

1. Query understanding: LLM expands and reformulates the user's query
2. Candidate retrieval: Traditional search (BM25 / Elasticsearch) finds initial candidates
3. AI reranking: LLM scores candidates by relevance to the original question
4. Answer generation: LLM synthesizes a direct answer from top results

Why AI Search Beats Keyword Search

Step 1: Query Expansion

from openai import OpenAI
client = OpenAI(base_url="https://api.aipower.me/v1", api_key="YOUR_KEY")

def expand_query(user_query):
    """Use LLM to generate better search queries."""
    response = client.chat.completions.create(
        model="deepseek/deepseek-chat",
        messages=[{
            "role": "user",
            "content": f"Generate 3 alternative search queries for: '{user_query}'\n"
                       "Return only the queries, one per line."
        }],
        max_tokens=100,
        temperature=0.3,
    )
    queries = response.choices[0].message.content.strip().split("\n")
    return [user_query] + queries  # Original + expansions

Step 2: AI Reranking

def rerank_results(query, candidates, top_k=5):
    """Use LLM to rerank search results by relevance."""
    candidates_text = "\n".join(
        f"[{i}] {doc['title']}: {doc['snippet'][:200]}"
        for i, doc in enumerate(candidates)
    )

    response = client.chat.completions.create(
        model="deepseek/deepseek-chat",
        messages=[{
            "role": "user",
            "content": f"Query: {query}\n\nDocuments:\n{candidates_text}\n\n"
                       f"Rank the top {top_k} most relevant document numbers. "
                       "Return only numbers separated by commas."
        }],
        max_tokens=50,
        temperature=0,
    )

    ranked_ids = [int(x.strip()) for x in response.choices[0].message.content.split(",")]
    return [candidates[i] for i in ranked_ids[:top_k]]

Step 3: Answer Generation

def generate_answer(query, top_results):
    """Generate a direct answer from search results."""
    context = "\n\n".join(
        f"Source: {doc['title']}\n{doc['content'][:500]}"
        for doc in top_results[:3]
    )

    response = client.chat.completions.create(
        model="deepseek/deepseek-chat",
        messages=[
            {"role": "system", "content": "Answer the question using the provided sources. "
                                          "Cite sources by title. Be concise."},
            {"role": "user", "content": f"Sources:\n{context}\n\nQuestion: {query}"},
        ],
    )
    return response.choices[0].message.content

Cost Analysis: AI Search

At $0.00055 per query, you can serve 1.8 million searches per $1,000 using DeepSeek through AIPower.

Build your AI search engine today at aipower.me — DeepSeek V3 at $0.34/M input tokens makes AI search economically viable at any scale. 50 free API calls to start.