<?xml version="1.0" encoding="utf-8"?>
<rss version="2.0" xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/">
    <channel>
        <title>Dheeraj Inampudi — Blog</title>
        <link>https://dheerajinampudi.com/blog</link>
        <description>Essays and notes on applied AI, products, engineering leadership, and building companies.</description>
        <lastBuildDate>Sun, 25 Oct 2026 00:00:00 GMT</lastBuildDate>
        <docs>https://validator.w3.org/feed/docs/rss2.html</docs>
        <generator>https://github.com/jpmonette/feed</generator>
        <language>en</language>
        <copyright>Copyright © 2026 Dheeraj Inampudi</copyright>
        <item>
            <title><![CDATA[Mental model: no free rent in your mind]]></title>
            <link>https://dheerajinampudi.com/blog/mental-models/no-free-rent-in-your-mind</link>
            <guid>https://dheerajinampudi.com/blog/mental-models/no-free-rent-in-your-mind</guid>
            <pubDate>Sun, 25 Oct 2026 00:00:00 GMT</pubDate>
            <description><![CDATA[If you wouldn't give a beautiful house to someone you dislike, why let them live rent-free in your head?]]></description>
            <content:encoded><![CDATA[<p>This one is for the people who keep replaying what someone said or did: the person who insulted you, betrayed you, or let you down, and somehow still takes up your evenings.</p>
<h2 class="anchor anchorTargetHideOnScrollNavbar_vjPI" id="picture-this">Picture this<a href="https://dheerajinampudi.com/blog/mental-models/no-free-rent-in-your-mind#picture-this" class="hash-link" aria-label="Direct link to Picture this" title="Direct link to Picture this" translate="no">​</a></h2>
<p>Imagine you're building a beautiful house.</p>
<p>You pay for it with money you worked hard for. You design every room carefully, choose the paintings, decorate every corner, and fill it with things you love. You put your time, your energy, and your heart into making it the best place you can imagine.</p>
<h2 class="anchor anchorTargetHideOnScrollNavbar_vjPI" id="the-question">The question<a href="https://dheerajinampudi.com/blog/mental-models/no-free-rent-in-your-mind#the-question" class="hash-link" aria-label="Direct link to The question" title="Direct link to The question" translate="no">​</a></h2>
<p>Now tell me something.</p>
<p><strong>Would you hand that house over, completely free, to someone you don't even like?</strong></p>
<p>Of course not.</p>
<p>So why let that same person live rent-free in your mind?</p>
<h2 class="anchor anchorTargetHideOnScrollNavbar_vjPI" id="whats-actually-happening">What's actually happening<a href="https://dheerajinampudi.com/blog/mental-models/no-free-rent-in-your-mind#whats-actually-happening" class="hash-link" aria-label="Direct link to What's actually happening" title="Direct link to What's actually happening" translate="no">​</a></h2>
<p>Every time I replay what they said, what they did, or how much I dislike them, I'm handing them a room in my head. Furnished. No rent.</p>
<p>And here's the part that stings: they're probably not thinking about me at all. I'm the one spending my time and energy on them.</p>
<p><strong>Your mind is far more precious than any house you could ever build.</strong></p>
<p>Letting go doesn't mean pretending it didn't happen, or that it didn't matter. It means I stop paying for it every day.</p>
<h2 class="anchor anchorTargetHideOnScrollNavbar_vjPI" id="the-takeaway">The takeaway<a href="https://dheerajinampudi.com/blog/mental-models/no-free-rent-in-your-mind#the-takeaway" class="hash-link" aria-label="Direct link to The takeaway" title="Direct link to The takeaway" translate="no">​</a></h2>
<p><strong>If you wouldn't give someone you dislike a beautiful house for free, why give them free space in your beautiful mind?</strong></p>
<hr>
<p><em>Part of my <a class="" href="https://dheerajinampudi.com/blog/tags/mental-models">Mental Models</a> series: small ideas I use to think more clearly, added one at a time.</em></p>]]></content:encoded>
            <category>Blog</category>
            <category>Mental Models</category>
        </item>
        <item>
            <title><![CDATA[Mental model: don't hand over the controls]]></title>
            <link>https://dheerajinampudi.com/blog/mental-models/dont-hand-over-the-controls</link>
            <guid>https://dheerajinampudi.com/blog/mental-models/dont-hand-over-the-controls</guid>
            <pubDate>Sat, 24 Oct 2026 00:00:00 GMT</pubDate>
            <description><![CDATA[When someone's words make you angry, disappointed, or ready to give up, ask who is really driving your next move.]]></description>
            <content:encoded><![CDATA[<p>This is one I come back to whenever a comment, a review, or a passing remark gets under my skin. It's for the moments when someone's words make you angry, leave you disappointed, or make you want to quit.</p>
<h2 class="anchor anchorTargetHideOnScrollNavbar_vjPI" id="the-situation">The situation<a href="https://dheerajinampudi.com/blog/mental-models/dont-hand-over-the-controls#the-situation" class="hash-link" aria-label="Direct link to The situation" title="Direct link to The situation" translate="no">​</a></h2>
<p>Someone says something. Maybe it's criticism, maybe it's careless, maybe it's meant to hurt. Before I've even thought about it, I'm angry, or discouraged, or already talking myself out of the thing I was about to do.</p>
<h2 class="anchor anchorTargetHideOnScrollNavbar_vjPI" id="think-about-it-this-way">Think about it this way<a href="https://dheerajinampudi.com/blog/mental-models/dont-hand-over-the-controls#think-about-it-this-way" class="hash-link" aria-label="Direct link to Think about it this way" title="Direct link to Think about it this way" translate="no">​</a></h2>
<p>If a few words from someone can instantly change how I feel and what I do next, what does that actually mean?</p>
<p>It means their words are holding the controls.</p>
<p>If someone can make me angry just by saying a sentence, haven't I quietly given them control over me?</p>
<h2 class="anchor anchorTargetHideOnScrollNavbar_vjPI" id="the-question">The question<a href="https://dheerajinampudi.com/blog/mental-models/dont-hand-over-the-controls#the-question" class="hash-link" aria-label="Direct link to The question" title="Direct link to The question" translate="no">​</a></h2>
<p><strong>Why would I give anyone that kind of power?</strong></p>
<p>It's my body, my mind, and my life. What I do should come from my own choices, not from whatever someone else happened to say.</p>
<p>People will say whatever they want. I can't control that, and I've stopped trying. What I <em>can</em> control is how I respond, what I do next, and whether I let their words stop me.</p>
<p>That doesn't mean ignoring feedback. If there's something true in what was said, I'll take it. But taking the useful part is a decision I make. Reacting on autopilot isn't.</p>
<h2 class="anchor anchorTargetHideOnScrollNavbar_vjPI" id="the-takeaway">The takeaway<a href="https://dheerajinampudi.com/blog/mental-models/dont-hand-over-the-controls#the-takeaway" class="hash-link" aria-label="Direct link to The takeaway" title="Direct link to The takeaway" translate="no">​</a></h2>
<p><strong>Never give someone else the power to control your actions. Your life should be driven by your will, not by their words.</strong></p>
<hr>
<p><em>Part of my <a class="" href="https://dheerajinampudi.com/blog/tags/mental-models">Mental Models</a> series: small ideas I use to think more clearly, added one at a time.</em></p>]]></content:encoded>
            <category>Blog</category>
            <category>Mental Models</category>
        </item>
        <item>
            <title><![CDATA[When to use machine learning]]></title>
            <link>https://dheerajinampudi.com/blog/when-to-use-machine-learning</link>
            <guid>https://dheerajinampudi.com/blog/when-to-use-machine-learning</guid>
            <pubDate>Thu, 29 Jan 2026 00:00:00 GMT</pubDate>
            <description><![CDATA[A practical decision sequence for choosing rules, pretrained models, or custom machine learning.]]></description>
            <content:encoded><![CDATA[<p>Machine learning is one of the most expensive ways to express a rule. It introduces uncertain output, evaluation work, data dependencies, and a system that changes when its inputs change. Those costs are justified when the problem contains patterns that ordinary software cannot express well.</p>
<p>I use three questions before a team commits to ML. The order matters because each question rules out a cheaper and more controllable approach.</p>
<h2 class="anchor anchorTargetHideOnScrollNavbar_vjPI" id="can-the-team-write-the-rules">Can the team write the rules?<a href="https://dheerajinampudi.com/blog/when-to-use-machine-learning#can-the-team-write-the-rules" class="hash-link" aria-label="Direct link to Can the team write the rules?" title="Direct link to Can the team write the rules?" translate="no">​</a></h2>
<p>If the logic can be stated, write it in software.</p>
<p>Tax calculations, eligibility checks, workflow states, thresholds, and deterministic validations usually belong in code. A rules-based implementation is easier to test, explain, and change. It also tells an operator exactly why the system acted.</p>
<p>This choice can feel less ambitious than an ML project. That is irrelevant. The job is to produce a dependable outcome with the least unnecessary machinery.</p>
<p>A useful test is to ask the domain expert to work through several cases. If the expert can name the inputs, decision steps, and exceptions with consistency, the first version should probably be deterministic.</p>
<h2 class="anchor anchorTargetHideOnScrollNavbar_vjPI" id="can-a-pretrained-model-recognize-the-pattern">Can a pretrained model recognize the pattern?<a href="https://dheerajinampudi.com/blog/when-to-use-machine-learning#can-a-pretrained-model-recognize-the-pattern" class="hash-link" aria-label="Direct link to Can a pretrained model recognize the pattern?" title="Direct link to Can a pretrained model recognize the pattern?" translate="no">​</a></h2>
<p>Some rules are hard to enumerate because they live in language, images, or a large number of small variations. A pretrained model may already handle the pattern well enough.</p>
<p>Common examples include:</p>
<ul>
<li class="">classifying the intent of a message</li>
<li class="">extracting fields from familiar documents</li>
<li class="">summarizing text for a person to review</li>
<li class="">answering a bounded question from supplied context</li>
<li class="">producing a draft that a person owns</li>
</ul>
<p>The model still needs evaluation. "The model can do this" is not an acceptance criterion. Build a set of representative inputs, define correct behavior, and inspect the failure cases that could cause harm.</p>
<p>Retrieval and tool calls do not increase a model's judgment. Retrieval supplies information. Tools give the model a way to act. The surrounding system still needs to decide what the model may do, which result is acceptable, and when to ask a person.</p>
<h2 class="anchor anchorTargetHideOnScrollNavbar_vjPI" id="is-the-useful-pattern-specific-to-your-data">Is the useful pattern specific to your data?<a href="https://dheerajinampudi.com/blog/when-to-use-machine-learning#is-the-useful-pattern-specific-to-your-data" class="hash-link" aria-label="Direct link to Is the useful pattern specific to your data?" title="Direct link to Is the useful pattern specific to your data?" translate="no">​</a></h2>
<p>Custom machine learning becomes a serious option when the rules are unknown, the pattern is absent from general training data, and the organization owns enough relevant examples.</p>
<p>Fraud detection, demand forecasting, equipment failure prediction, recommendation systems, and domain-specific risk scoring often fit this shape. The signal appears across many examples and is difficult to encode by hand.</p>
<p>Before training, check whether the data describes the decision you need to make. Volume cannot repair the wrong label, a biased collection process, or a target that arrives after the decision point. A smaller dataset with a sound definition can be more useful than a large historical dump.</p>
<p>Custom ML also creates an operating commitment. Someone must monitor drift, retrain when the environment changes, investigate regressions, and decide when the model should stop acting.</p>
<h2 class="anchor anchorTargetHideOnScrollNavbar_vjPI" id="the-decision-in-one-sequence">The decision in one sequence<a href="https://dheerajinampudi.com/blog/when-to-use-machine-learning#the-decision-in-one-sequence" class="hash-link" aria-label="Direct link to The decision in one sequence" title="Direct link to The decision in one sequence" translate="no">​</a></h2>
<p>Use this order:</p>
<ol>
<li class="">Write deterministic rules when the decision can be specified.</li>
<li class="">Evaluate a pretrained model when the pattern is common but difficult to enumerate.</li>
<li class="">Train a custom model when the useful signal is specific to your data and simpler approaches fail.</li>
</ol>
<p>Hybrid systems are common. A model may extract intent while deterministic code applies policy. A custom score may prioritize work while a person makes the final decision. The boundaries matter more than the label attached to the system.</p>
<h2 class="anchor anchorTargetHideOnScrollNavbar_vjPI" id="what-engineers-still-own">What engineers still own<a href="https://dheerajinampudi.com/blog/when-to-use-machine-learning#what-engineers-still-own" class="hash-link" aria-label="Direct link to What engineers still own" title="Direct link to What engineers still own" translate="no">​</a></h2>
<p>AI can produce a large share of implementation code. Engineers still own the choice of method, the test set, the failure policy, and the result in production.</p>
<p>Good applied AI work begins by refusing unnecessary uncertainty. Add a model after the system has earned the complexity.</p>]]></content:encoded>
            <category>Blog</category>
            <category>AI Systems</category>
            <category>Product Decisions</category>
        </item>
        <item>
            <title><![CDATA[Leadership principles I practice]]></title>
            <link>https://dheerajinampudi.com/blog/leadership-principles</link>
            <guid>https://dheerajinampudi.com/blog/leadership-principles</guid>
            <pubDate>Wed, 28 Jan 2026 00:00:00 GMT</pubDate>
            <description><![CDATA[Notes on clarity, ownership, and leading engineering teams when code is cheap and judgment is not.]]></description>
            <content:encoded><![CDATA[<p>Leadership became more concrete for me as my responsibility moved beyond the code. A technical decision changes a system. A leadership decision changes who can make the next hundred decisions, and how well they make them.</p>
<p>These are the principles I return to while building products and teams.</p>
<h2 class="anchor anchorTargetHideOnScrollNavbar_vjPI" id="plan-while-the-direction-is-uncertain">Plan while the direction is uncertain<a href="https://dheerajinampudi.com/blog/leadership-principles#plan-while-the-direction-is-uncertain" class="hash-link" aria-label="Direct link to Plan while the direction is uncertain" title="Direct link to Plan while the direction is uncertain" translate="no">​</a></h2>
<p>Planning should reduce uncertainty. Once the direction is clear, another planning cycle often delays the evidence the team needs.</p>
<p>I plan when the team disagrees about the goal, the cost of reversal is high, or an untested assumption sits under the work. I push for delivery when the decision is reversible and customer use will teach us more than another meeting.</p>
<p>The leader's job is to tell those conditions apart and make the change of pace explicit.</p>
<h2 class="anchor anchorTargetHideOnScrollNavbar_vjPI" id="adapt-to-the-person-and-hold-the-outcome-steady">Adapt to the person and hold the outcome steady<a href="https://dheerajinampudi.com/blog/leadership-principles#adapt-to-the-person-and-hold-the-outcome-steady" class="hash-link" aria-label="Direct link to Adapt to the person and hold the outcome steady" title="Direct link to Adapt to the person and hold the outcome steady" translate="no">​</a></h2>
<p>People need different kinds of support. One engineer may need a clear boundary and room to work. Another may need faster feedback or help turning a broad problem into a first step.</p>
<p>Adapting the support does not require lowering the standard. The expected outcome, owner, and review point should remain clear. Fair leadership gives people what they need to meet a shared bar.</p>
<h2 class="anchor anchorTargetHideOnScrollNavbar_vjPI" id="ai-can-produce-code-people-own-the-result">AI can produce code; people own the result<a href="https://dheerajinampudi.com/blog/leadership-principles#ai-can-produce-code-people-own-the-result" class="hash-link" aria-label="Direct link to AI can produce code; people own the result" title="Direct link to AI can produce code; people own the result" translate="no">​</a></h2>
<p>Generated code makes implementation faster. It does not own intent, judgment, coordination, or the production incident.</p>
<p>Engineering leadership now places more weight on problem selection and review. Teams need to understand why a change exists, which behavior matters, how the system can fail, and who will make the call when the evidence is mixed.</p>
<p>Code volume is a weak measure of progress. A team that deletes an unnecessary service or replaces a model with a rule may have made the stronger engineering decision.</p>
<h2 class="anchor anchorTargetHideOnScrollNavbar_vjPI" id="put-company-goals-into-the-work">Put company goals into the work<a href="https://dheerajinampudi.com/blog/leadership-principles#put-company-goals-into-the-work" class="hash-link" aria-label="Direct link to Put company goals into the work" title="Direct link to Put company goals into the work" translate="no">​</a></h2>
<p>Company goals become useful when teams can connect them to product behavior and customer outcomes. A slogan at an all-hands does not create alignment.</p>
<p>I want a team to know which customer problem a project addresses, what evidence will change our view, and which tradeoff we accepted. That context helps people make local decisions without waiting for approval.</p>
<p>Personal growth should happen through meaningful responsibility. The company gets better work, and the person gains judgment that survives the current project.</p>
<h2 class="anchor anchorTargetHideOnScrollNavbar_vjPI" id="reward-care-and-ownership">Reward care and ownership<a href="https://dheerajinampudi.com/blog/leadership-principles#reward-care-and-ownership" class="hash-link" aria-label="Direct link to Reward care and ownership" title="Direct link to Reward care and ownership" translate="no">​</a></h2>
<p>Technical skill can grow through practice. Care is harder to create through process.</p>
<p>I look for people who notice when a result is incomplete, ask about the effect on customers, and stay with a problem through the unglamorous last mile. Leaders should protect that behavior from environments that reward visible activity over durable outcomes.</p>
<p>Ownership also needs boundaries. A person cannot own an outcome without authority, context, and time to act. Assigning responsibility without those conditions is delegation in name only.</p>
<h2 class="anchor anchorTargetHideOnScrollNavbar_vjPI" id="do-the-homework-before-asking-for-trust">Do the homework before asking for trust<a href="https://dheerajinampudi.com/blog/leadership-principles#do-the-homework-before-asking-for-trust" class="hash-link" aria-label="Direct link to Do the homework before asking for trust" title="Direct link to Do the homework before asking for trust" translate="no">​</a></h2>
<p>Teams read a leader's clarity as evidence of preparation. A vague answer may be honest, but repeated vagueness tells people that the decision has no owner.</p>
<p>When I do not have an answer, I say what I need to learn and when I will return with a position. That is more useful than filling the space with confidence.</p>
<p>The quality of a team's decisions cannot rise far above the quality of the context its leaders provide. Clear goals, named tradeoffs, and honest uncertainty give people something solid to work with.</p>]]></content:encoded>
            <category>Blog</category>
            <category>Leadership</category>
            <category>Founder Notes</category>
        </item>
        <item>
            <title><![CDATA[LLM architecture mental models]]></title>
            <link>https://dheerajinampudi.com/blog/llm-architecture-mental-models</link>
            <guid>https://dheerajinampudi.com/blog/llm-architecture-mental-models</guid>
            <pubDate>Mon, 26 Jan 2026 00:00:00 GMT</pubDate>
            <description><![CDATA[A field guide to model selection through context, attention, activation cost, and failure behavior.]]></description>
            <content:encoded><![CDATA[<p>Model selection begins with the workload. Benchmarks and parameter counts help only after the team defines the input, required behavior, latency budget, deployment constraints, and failures it can tolerate.</p>
<p>These mental models help application engineers read a model card and turn architecture details into production questions.</p>
<h2 class="anchor anchorTargetHideOnScrollNavbar_vjPI" id="dense-and-mixture-of-experts-models">Dense and mixture-of-experts models<a href="https://dheerajinampudi.com/blog/llm-architecture-mental-models#dense-and-mixture-of-experts-models" class="hash-link" aria-label="Direct link to Dense and mixture-of-experts models" title="Direct link to Dense and mixture-of-experts models" translate="no">​</a></h2>
<p>A dense model uses the same parameter set for each token. A mixture-of-experts model routes tokens through a subset of specialized feed-forward blocks.</p>
<p>The distinction changes how to read size. Total parameters describe stored capacity. Activated parameters give a better signal for compute per token. A large MoE model can activate far fewer parameters than its headline number suggests, though it may still require substantial memory and more complex serving.</p>
<p>Ask:</p>
<ul>
<li class="">How many parameters are active for each token?</li>
<li class="">Can the serving stack place and route the experts efficiently?</li>
<li class="">Does routing behavior remain stable on the target workload?</li>
<li class="">Is the latency distribution acceptable, not only the average?</li>
</ul>
<p>Dense models often offer simpler deployment behavior. MoE models can offer a useful quality-to-compute tradeoff when the serving system supports them well.</p>
<h2 class="anchor anchorTargetHideOnScrollNavbar_vjPI" id="context-is-capacity-not-memory">Context is capacity, not memory<a href="https://dheerajinampudi.com/blog/llm-architecture-mental-models#context-is-capacity-not-memory" class="hash-link" aria-label="Direct link to Context is capacity, not memory" title="Direct link to Context is capacity, not memory" translate="no">​</a></h2>
<p>A context window states how many tokens a model can accept. It does not promise equal attention to every part of that input, reliable retrieval, or durable memory between calls.</p>
<p>Long context affects cost and latency through attention computation and the key-value cache. It also changes evaluation. A model can accept a long document and still miss a fact in the middle, combine unrelated passages, or follow an instruction from the wrong source.</p>
<p>Ask:</p>
<ul>
<li class="">How much context does the real task need?</li>
<li class="">Which information must remain exact?</li>
<li class="">Can retrieval reduce the input before generation?</li>
<li class="">Does the model find and use evidence at the positions your application creates?</li>
</ul>
<p>Treat persistent memory as a separate system. It needs rules for what to store, how to retrieve it, when to expire it, and how a user can correct it.</p>
<h2 class="anchor anchorTargetHideOnScrollNavbar_vjPI" id="attention-design-shapes-long-context-cost">Attention design shapes long-context cost<a href="https://dheerajinampudi.com/blog/llm-architecture-mental-models#attention-design-shapes-long-context-cost" class="hash-link" aria-label="Direct link to Attention design shapes long-context cost" title="Direct link to Attention design shapes long-context cost" translate="no">​</a></h2>
<p>Architectures use techniques such as grouped-query attention, multi-query attention, sliding windows, and optimized attention kernels to reduce the cost of long inputs and decoding.</p>
<p>The model specification alone does not determine application performance. Hardware, serving framework, batch behavior, prompt length, output length, and cache reuse all affect the result.</p>
<p>Measure the full request path with representative traffic. A theoretical context limit has little value if the target latency or concurrency collapses long before that limit.</p>
<h2 class="anchor anchorTargetHideOnScrollNavbar_vjPI" id="tokenization-is-part-of-the-bill">Tokenization is part of the bill<a href="https://dheerajinampudi.com/blog/llm-architecture-mental-models#tokenization-is-part-of-the-bill" class="hash-link" aria-label="Direct link to Tokenization is part of the bill" title="Direct link to Tokenization is part of the bill" translate="no">​</a></h2>
<p>Models do not read characters or words directly. Their tokenizer can represent the same source text with different token counts.</p>
<p>This matters for code, identifiers, multilingual content, structured records, and specialized terminology. More tokens increase input cost and consume context capacity. Token boundaries can also affect extraction and constrained output.</p>
<p>Run the tokenizer against real inputs before estimating cost. Averages from general English text will not describe every product.</p>
<h2 class="anchor anchorTargetHideOnScrollNavbar_vjPI" id="quantization-trades-precision-for-operating-cost">Quantization trades precision for operating cost<a href="https://dheerajinampudi.com/blog/llm-architecture-mental-models#quantization-trades-precision-for-operating-cost" class="hash-link" aria-label="Direct link to Quantization trades precision for operating cost" title="Direct link to Quantization trades precision for operating cost" translate="no">​</a></h2>
<p>Quantization stores or computes model values with fewer bits. It can reduce memory use and increase throughput, but the effect on quality depends on the model, method, hardware, and task.</p>
<p>Small regressions can matter in tool arguments, structured output, numerical work, or rare edge cases even when general text still reads well.</p>
<p>Evaluate the quantized build against the same acceptance set as the original model. Compare failure categories and latency rather than relying on a single aggregate score.</p>
<h2 class="anchor anchorTargetHideOnScrollNavbar_vjPI" id="post-training-defines-behavior">Post-training defines behavior<a href="https://dheerajinampudi.com/blog/llm-architecture-mental-models#post-training-defines-behavior" class="hash-link" aria-label="Direct link to Post-training defines behavior" title="Direct link to Post-training defines behavior" translate="no">​</a></h2>
<p>Two models with similar base architecture can behave differently after instruction tuning, preference optimization, safety training, and tool-use training.</p>
<p>Application engineers experience this as behavior: whether the model follows a schema, calls the right tool, asks for missing information, cites supplied evidence, and refuses outside the intended boundary.</p>
<p>Prompt work cannot repair every post-training mismatch. A smaller model trained for the required interaction may outperform a larger general model in a bounded workflow.</p>
<h2 class="anchor anchorTargetHideOnScrollNavbar_vjPI" id="failure-behavior-beats-the-leaderboard">Failure behavior beats the leaderboard<a href="https://dheerajinampudi.com/blog/llm-architecture-mental-models#failure-behavior-beats-the-leaderboard" class="hash-link" aria-label="Direct link to Failure behavior beats the leaderboard" title="Direct link to Failure behavior beats the leaderboard" translate="no">​</a></h2>
<p>Average benchmark performance hides the cases that determine production trust.</p>
<p>Build an evaluation set around the system's decisions. Include ambiguous requests, missing data, conflicting context, long inputs, invalid tool results, and requests that should reach a person. Record how the model fails and whether the surrounding system can detect the failure.</p>
<p>The best model for an application is often the one whose errors the team can recognize and contain.</p>
<h2 class="anchor anchorTargetHideOnScrollNavbar_vjPI" id="keep-a-model-decision-record">Keep a model decision record<a href="https://dheerajinampudi.com/blog/llm-architecture-mental-models#keep-a-model-decision-record" class="hash-link" aria-label="Direct link to Keep a model decision record" title="Direct link to Keep a model decision record" translate="no">​</a></h2>
<p>For each candidate, record:</p>
<ol>
<li class="">The workload and representative inputs</li>
<li class="">Required output behavior</li>
<li class="">Context and tokenizer measurements</li>
<li class="">Latency, throughput, and cost under expected traffic</li>
<li class="">Failure categories and escalation policy</li>
<li class="">Security, privacy, and deployment constraints</li>
</ol>
<p>That record turns model selection into an engineering decision. It also makes the next evaluation faster when models, prices, and product requirements change.</p>]]></content:encoded>
            <category>Blog</category>
            <category>AI Systems</category>
            <category>Production AI</category>
        </item>
        <item>
            <title><![CDATA[LLM evaluation guide for AI engineers]]></title>
            <link>https://dheerajinampudi.com/blog/spaces/ai-space/llm-evaluation-guide</link>
            <guid>https://dheerajinampudi.com/blog/spaces/ai-space/llm-evaluation-guide</guid>
            <pubDate>Mon, 26 Jan 2026 00:00:00 GMT</pubDate>
            <description><![CDATA[A parameter-by-parameter checklist for evaluating LLMs in production, from architecture and context to serving, tool use and reliability.]]></description>
            <content:encoded><![CDATA[<p>As an application AI engineer, you don't need to train LLMs—but you <strong>must understand how they're built</strong> to make correct architectural decisions. Think of it like understanding database internals without writing a storage engine.</p>
<p>This guide consolidates the critical parameters you need to evaluate when selecting, deploying, or debugging LLMs in production.</p>
<p><img decoding="async" loading="lazy" alt="LLM evaluation framework infographic: eight areas around a core evaluation framework — model architecture, context and position, MoE experts, tokenization, training and alignment, inference and serving, agent and tools, and production metrics" src="https://dheerajinampudi.com/assets/images/llm_evaluation_infographic-42eb1b997544b688d9fbab53cc58689f.png" width="1024" height="1024" class="img_ev3q"></p>
<hr>
<h2 class="anchor anchorTargetHideOnScrollNavbar_vjPI" id="model-architecture">Model architecture<a href="https://dheerajinampudi.com/blog/spaces/ai-space/llm-evaluation-guide#model-architecture" class="hash-link" aria-label="Direct link to Model architecture" title="Direct link to Model architecture" translate="no">​</a></h2>
<h3 class="anchor anchorTargetHideOnScrollNavbar_vjPI" id="model-type">Model type<a href="https://dheerajinampudi.com/blog/spaces/ai-space/llm-evaluation-guide#model-type" class="hash-link" aria-label="Direct link to Model type" title="Direct link to Model type" translate="no">​</a></h3>
<table><thead><tr><th>Aspect</th><th>Details</th></tr></thead><tbody><tr><td><strong>What it is</strong></td><td>Next-token prediction (left-to-right) generator</td></tr><tr><td><strong>Key questions</strong></td><td>Is it decoder-only? Any special heads for classification/tooling?</td></tr><tr><td><strong>Why it matters</strong></td><td>Determines best-fit tasks (generation vs embedding) and streaming behavior</td></tr><tr><td><strong>Common options</strong></td><td>Decoder-only (GPT/LLaMA/Qwen), Encoder-decoder (T5), Encoder-only (BERT)</td></tr><tr><td><strong>What you control</strong></td><td>Prompting pattern, stop sequences, structured output constraints</td></tr><tr><td><strong>Tradeoffs</strong></td><td>Decoder-only can hallucinate; not ideal alone for retrieval/classification</td></tr><tr><td><strong>Smell test</strong></td><td>Code assistant or refactor agent → decoder-only is the right choice</td></tr></tbody></table>
<h3 class="anchor anchorTargetHideOnScrollNavbar_vjPI" id="parameters-total-vs-activated-moe">Parameters: total vs activated (MoE)<a href="https://dheerajinampudi.com/blog/spaces/ai-space/llm-evaluation-guide#parameters-total-vs-activated-moe" class="hash-link" aria-label="Direct link to Parameters: total vs activated (MoE)" title="Direct link to Parameters: total vs activated (MoE)" translate="no">​</a></h3>
<table><thead><tr><th>Aspect</th><th>Details</th></tr></thead><tbody><tr><td><strong>What it is</strong></td><td>Stored capacity vs per-token compute actually used</td></tr><tr><td><strong>Key questions</strong></td><td>Is it MoE? What's the activated compute per token?</td></tr><tr><td><strong>Why it matters</strong></td><td>Cost/latency can be close to a small model with large-model quality</td></tr><tr><td><strong>Common options</strong></td><td>Dense, MoE (top-k), Hybrid</td></tr><tr><td><strong>What you control</strong></td><td>Batch sizing, expert parallelism, routing config (if exposed)</td></tr><tr><td><strong>Tradeoffs</strong></td><td>MoE can have load imbalance; routing instability; harder to serve</td></tr><tr><td><strong>Smell test</strong></td><td>"30B total, ~3B active" → cheap inference, big VRAM footprint</td></tr></tbody></table>
<h3 class="anchor anchorTargetHideOnScrollNavbar_vjPI" id="depth-and-width">Depth and width<a href="https://dheerajinampudi.com/blog/spaces/ai-space/llm-evaluation-guide#depth-and-width" class="hash-link" aria-label="Direct link to Depth and width" title="Direct link to Depth and width" translate="no">​</a></h3>
<table><thead><tr><th>Parameter</th><th>What it is</th><th>Why it matters</th><th>Tradeoffs</th></tr></thead><tbody><tr><td><strong>Layers (Depth)</strong></td><td>Transformer block count (24, 32, 48, 80+)</td><td>Correlates with multi-step reasoning and abstraction</td><td>Deeper = higher latency; harder to quantize aggressively</td></tr><tr><td><strong>Hidden size (Width)</strong></td><td>Representation capacity per layer (2k–10k+)</td><td>Affects richness of representations + code reasoning</td><td>Wider increases VRAM + compute; may not help if data mix is weak</td></tr></tbody></table>
<h3 class="anchor anchorTargetHideOnScrollNavbar_vjPI" id="attention-architecture">Attention architecture<a href="https://dheerajinampudi.com/blog/spaces/ai-space/llm-evaluation-guide#attention-architecture" class="hash-link" aria-label="Direct link to Attention architecture" title="Direct link to Attention architecture" translate="no">​</a></h3>
<table><thead><tr><th>Aspect</th><th>Details</th></tr></thead><tbody><tr><td><strong>What it is</strong></td><td>Attention parallelism mechanism; GQA/MQA reduces KV cache size</td></tr><tr><td><strong>Key questions</strong></td><td>Is it MHA, GQA, or MQA? What's the Q<!-- -->:KV<!-- --> head ratio?</td></tr><tr><td><strong>Why it matters</strong></td><td>Big driver of long-context feasibility and throughput</td></tr><tr><td><strong>Common options</strong></td><td>MHA (Multi-Head), GQA (Grouped Query), MQA (Single KV)</td></tr><tr><td><strong>What you control</strong></td><td>KV cache precision, batching, context window policy</td></tr><tr><td><strong>Tradeoffs</strong></td><td>Too aggressive KV sharing can slightly reduce quality</td></tr><tr><td><strong>Smell test</strong></td><td>GQA (32Q/4KV) → much smaller KV cache for long context</td></tr></tbody></table>
<hr>
<h2 class="anchor anchorTargetHideOnScrollNavbar_vjPI" id="context-and-positioning">Context and positioning<a href="https://dheerajinampudi.com/blog/spaces/ai-space/llm-evaluation-guide#context-and-positioning" class="hash-link" aria-label="Direct link to Context and positioning" title="Direct link to Context and positioning" translate="no">​</a></h2>
<h3 class="anchor anchorTargetHideOnScrollNavbar_vjPI" id="context-length">Context length<a href="https://dheerajinampudi.com/blog/spaces/ai-space/llm-evaluation-guide#context-length" class="hash-link" aria-label="Direct link to Context length" title="Direct link to Context length" translate="no">​</a></h3>
<table><thead><tr><th>Aspect</th><th>Details</th></tr></thead><tbody><tr><td><strong>What it is</strong></td><td>Maximum tokens handled <em>as trained</em></td></tr><tr><td><strong>Key questions</strong></td><td>Native or RoPE-scaled? What's quality past 32k/64k?</td></tr><tr><td><strong>Why it matters</strong></td><td>Long-context correctness is non-linear; "supports" ≠ "understands"</td></tr><tr><td><strong>Common options</strong></td><td>8k / 16k / 32k / 128k / 256k+</td></tr><tr><td><strong>What you control</strong></td><td>Chunking, sliding window, retrieval strategy, summarization</td></tr><tr><td><strong>Tradeoffs</strong></td><td>KV cache explosion; "lost in the middle" phenomenon; higher latency</td></tr><tr><td><strong>Smell test</strong></td><td>If it forgets key constraints in long prompts → long-context weakness</td></tr></tbody></table>
<h3 class="anchor anchorTargetHideOnScrollNavbar_vjPI" id="positional-encoding">Positional encoding<a href="https://dheerajinampudi.com/blog/spaces/ai-space/llm-evaluation-guide#positional-encoding" class="hash-link" aria-label="Direct link to Positional encoding" title="Direct link to Positional encoding" translate="no">​</a></h3>
<table><thead><tr><th>Aspect</th><th>Details</th></tr></thead><tbody><tr><td><strong>What it is</strong></td><td>How the model represents token positions</td></tr><tr><td><strong>Key questions</strong></td><td>RoPE? ALiBi? Any long-context scaling (NTK/YaRN)?</td></tr><tr><td><strong>Why it matters</strong></td><td>Governs long-range generalization</td></tr><tr><td><strong>Common options</strong></td><td>RoPE, ALiBi, Absolute, RoPE scaling variants</td></tr><tr><td><strong>What you control</strong></td><td>Prompt structure (headings), chunking order, retrieval placement</td></tr><tr><td><strong>Tradeoffs</strong></td><td>Scaling hacks can break beyond the trained window</td></tr><tr><td><strong>Smell test</strong></td><td>Native 256k trained usually beats RoPE-hacked 256k</td></tr></tbody></table>
<hr>
<h2 class="anchor anchorTargetHideOnScrollNavbar_vjPI" id="mixture-of-experts-moe">Mixture of experts (MoE)<a href="https://dheerajinampudi.com/blog/spaces/ai-space/llm-evaluation-guide#mixture-of-experts-moe" class="hash-link" aria-label="Direct link to Mixture of experts (MoE)" title="Direct link to Mixture of experts (MoE)" translate="no">​</a></h2>
<blockquote>
<p>Only applicable to MoE models (e.g., Mixtral, Qwen-MoE, DeepSeek-V2)</p>
</blockquote>
<table><thead><tr><th>Parameter</th><th>What it is</th><th>Key questions</th><th>Tradeoffs</th></tr></thead><tbody><tr><td><strong># Experts</strong></td><td>Count of expert FFNs (8–128+)</td><td>How many? Are they specialized (code/math)?</td><td>Bigger memory footprint; more complex routing</td></tr><tr><td><strong>Activated experts (top-k)</strong></td><td>How many experts run per token</td><td>k=2/4/8? Does it vary?</td><td>Higher k = better quality, more compute; tail latency spikes</td></tr><tr><td><strong>Router / load balancing</strong></td><td>Mechanism selecting experts</td><td>Token-level routing? Load balance loss?</td><td>"Hot" experts cause congestion; quality variance</td></tr></tbody></table>
<hr>
<h2 class="anchor anchorTargetHideOnScrollNavbar_vjPI" id="tokenization-and-internals">Tokenization and internals<a href="https://dheerajinampudi.com/blog/spaces/ai-space/llm-evaluation-guide#tokenization-and-internals" class="hash-link" aria-label="Direct link to Tokenization and internals" title="Direct link to Tokenization and internals" translate="no">​</a></h2>
<table><thead><tr><th>Parameter</th><th>What it is</th><th>Key questions</th><th>Engineering impact</th></tr></thead><tbody><tr><td><strong>Tokenizer + vocab</strong></td><td>How text/code becomes tokens</td><td>BPE/SentencePiece? Code-optimized? Multilingual?</td><td>Bad tokenizer → 20–40% extra tokens; worse code formatting</td></tr><tr><td><strong>Activation function</strong></td><td>Non-linearity in FFN layers</td><td>SwiGLU? GeGLU? ReLU?</td><td>Some activations quantize better than others</td></tr></tbody></table>
<hr>
<h2 class="anchor anchorTargetHideOnScrollNavbar_vjPI" id="training-and-alignment">Training and alignment<a href="https://dheerajinampudi.com/blog/spaces/ai-space/llm-evaluation-guide#training-and-alignment" class="hash-link" aria-label="Direct link to Training and alignment" title="Direct link to Training and alignment" translate="no">​</a></h2>
<h3 class="anchor anchorTargetHideOnScrollNavbar_vjPI" id="training-stage">Training stage<a href="https://dheerajinampudi.com/blog/spaces/ai-space/llm-evaluation-guide#training-stage" class="hash-link" aria-label="Direct link to Training stage" title="Direct link to Training stage" translate="no">​</a></h3>
<table><thead><tr><th>Aspect</th><th>Details</th></tr></thead><tbody><tr><td><strong>What it is</strong></td><td>Pretrain gives capability; post-train adds instruction-following/alignment</td></tr><tr><td><strong>Key questions</strong></td><td>How much instruction data? RLHF? Tool-use training? Safety tuning?</td></tr><tr><td><strong>Why it matters</strong></td><td>"Usable in prod" depends more on post-train than size</td></tr><tr><td><strong>Common options</strong></td><td>SFT only, RLHF/DPO/IPO, Tool-use finetune, Safety filters</td></tr><tr><td><strong>What you control</strong></td><td>System prompt, policy layer, tool schema, eval harness</td></tr><tr><td><strong>Tradeoffs</strong></td><td>Over-alignment/refusals; shallow SFT can be prompt-fragile</td></tr><tr><td><strong>Smell test</strong></td><td>If it ignores instructions or breaks JSON easily → weak post-train</td></tr></tbody></table>
<h3 class="anchor anchorTargetHideOnScrollNavbar_vjPI" id="training-data-mix">Training data mix<a href="https://dheerajinampudi.com/blog/spaces/ai-space/llm-evaluation-guide#training-data-mix" class="hash-link" aria-label="Direct link to Training data mix" title="Direct link to Training data mix" translate="no">​</a></h3>
<table><thead><tr><th>Aspect</th><th>Details</th></tr></thead><tbody><tr><td><strong>What it is</strong></td><td>Ratio of code/text/math + freshness</td></tr><tr><td><strong>Key questions</strong></td><td>How much code? Which languages? License-clean?</td></tr><tr><td><strong>Why it matters</strong></td><td>Strong predictor of code quality and hallucination rate</td></tr><tr><td><strong>Common options</strong></td><td>Code-heavy, Balanced, Math-heavy</td></tr><tr><td><strong>Tradeoffs</strong></td><td>Data contamination/licensing risk; stale frameworks</td></tr><tr><td><strong>Smell test</strong></td><td>If it's bad at modern React/TypeScript patterns → stale data mix</td></tr></tbody></table>
<hr>
<h2 class="anchor anchorTargetHideOnScrollNavbar_vjPI" id="inference-and-serving">Inference and serving<a href="https://dheerajinampudi.com/blog/spaces/ai-space/llm-evaluation-guide#inference-and-serving" class="hash-link" aria-label="Direct link to Inference and serving" title="Direct link to Inference and serving" translate="no">​</a></h2>
<h3 class="anchor anchorTargetHideOnScrollNavbar_vjPI" id="precision-and-quantization">Precision and quantization<a href="https://dheerajinampudi.com/blog/spaces/ai-space/llm-evaluation-guide#precision-and-quantization" class="hash-link" aria-label="Direct link to Precision and quantization" title="Direct link to Precision and quantization" translate="no">​</a></h3>
<table><thead><tr><th>Parameter</th><th>What it is</th><th>Key questions</th><th>Tradeoffs</th></tr></thead><tbody><tr><td><strong>Precision</strong></td><td>Numerical format (FP32/BF16/FP16/FP8)</td><td>FP16 vs BF16 vs FP8? Mixed precision?</td><td>FP16 can overflow; FP8 needs calibration</td></tr><tr><td><strong>Quantization</strong></td><td>Lower-bit weights (INT8/INT4)</td><td>GPTQ/AWQ? KV quantization?</td><td>Biggest lever for $/token but can break math/JSON/tool calls</td></tr></tbody></table>
<h3 class="anchor anchorTargetHideOnScrollNavbar_vjPI" id="attention-and-memory">Attention and memory<a href="https://dheerajinampudi.com/blog/spaces/ai-space/llm-evaluation-guide#attention-and-memory" class="hash-link" aria-label="Direct link to Attention and memory" title="Direct link to Attention and memory" translate="no">​</a></h3>
<table><thead><tr><th>Parameter</th><th>What it is</th><th>Key questions</th><th>Why it matters</th></tr></thead><tbody><tr><td><strong>Attention kernel</strong></td><td>GPU attention algorithm</td><td>FlashAttention2/3? Paged attention?</td><td>Can mean 2–4× throughput difference</td></tr><tr><td><strong>KV cache strategy</strong></td><td>How past tokens' K/V are stored</td><td>KV precision? Paged KV? Prefix caching?</td><td>Dominant factor for long-context serving economics</td></tr></tbody></table>
<h3 class="anchor anchorTargetHideOnScrollNavbar_vjPI" id="batching-and-parallelism">Batching and parallelism<a href="https://dheerajinampudi.com/blog/spaces/ai-space/llm-evaluation-guide#batching-and-parallelism" class="hash-link" aria-label="Direct link to Batching and parallelism" title="Direct link to Batching and parallelism" translate="no">​</a></h3>
<table><thead><tr><th>Parameter</th><th>Details</th></tr></thead><tbody><tr><td><strong>Batching strategy</strong></td><td>Static vs continuous batching; drives throughput and cost</td></tr><tr><td><strong>Parallelism (TP/PP/EP)</strong></td><td>Tensor/Pipeline/Expert parallelism; determines scaling + infra cost</td></tr></tbody></table>
<h3 class="anchor anchorTargetHideOnScrollNavbar_vjPI" id="decoding-parameters">Decoding parameters<a href="https://dheerajinampudi.com/blog/spaces/ai-space/llm-evaluation-guide#decoding-parameters" class="hash-link" aria-label="Direct link to Decoding parameters" title="Direct link to Decoding parameters" translate="no">​</a></h3>
<table><thead><tr><th>Aspect</th><th>Details</th></tr></thead><tbody><tr><td><strong>What it is</strong></td><td>Sampling parameters for generation</td></tr><tr><td><strong>Key questions</strong></td><td>Temperature/top-p/top-k? Repetition penalty?</td></tr><tr><td><strong>Common profiles</strong></td><td>Code mode: temp 0–0.2, low top-p; Creative: temp 0.7–1.0</td></tr><tr><td><strong>Tradeoffs</strong></td><td>Too creative → wrong code; too greedy → brittle phrasing</td></tr></tbody></table>
<hr>
<h2 class="anchor anchorTargetHideOnScrollNavbar_vjPI" id="agent-and-tool-use">Agent and tool use<a href="https://dheerajinampudi.com/blog/spaces/ai-space/llm-evaluation-guide#agent-and-tool-use" class="hash-link" aria-label="Direct link to Agent and tool use" title="Direct link to Agent and tool use" translate="no">​</a></h2>
<h3 class="anchor anchorTargetHideOnScrollNavbar_vjPI" id="tool-and-function-calling">Tool and function calling<a href="https://dheerajinampudi.com/blog/spaces/ai-space/llm-evaluation-guide#tool-and-function-calling" class="hash-link" aria-label="Direct link to Tool and function calling" title="Direct link to Tool and function calling" translate="no">​</a></h3>
<table><thead><tr><th>Aspect</th><th>Details</th></tr></thead><tbody><tr><td><strong>What it is</strong></td><td>Whether model learned tool usage</td></tr><tr><td><strong>Key questions</strong></td><td>Native tool-calling? Or prompt-hacked?</td></tr><tr><td><strong>Why it matters</strong></td><td>Determines agent reliability</td></tr><tr><td><strong>Common options</strong></td><td>Trained function calling, XML tags, Prompt-only</td></tr><tr><td><strong>What you control</strong></td><td>Tool schema design, retries, self-heal loops</td></tr><tr><td><strong>Smell test</strong></td><td>If it keeps "thinking" instead of calling a tool → weak tool training</td></tr></tbody></table>
<h3 class="anchor anchorTargetHideOnScrollNavbar_vjPI" id="structured-output">Structured output<a href="https://dheerajinampudi.com/blog/spaces/ai-space/llm-evaluation-guide#structured-output" class="hash-link" aria-label="Direct link to Structured output" title="Direct link to Structured output" translate="no">​</a></h3>
<table><thead><tr><th>Aspect</th><th>Details</th></tr></thead><tbody><tr><td><strong>What it is</strong></td><td>Ability to emit valid JSON/DSL</td></tr><tr><td><strong>Key questions</strong></td><td>Does it support JSON schema / constrained decoding?</td></tr><tr><td><strong>Why it matters</strong></td><td>Critical for agents and tool calls</td></tr><tr><td><strong>Common options</strong></td><td>Free-form, JSON mode, CFG/constrained decoding</td></tr><tr><td><strong>What you control</strong></td><td>Use constrained decoding, schema validation + retries</td></tr><tr><td><strong>Smell test</strong></td><td>If tool calls break 5% of the time → production pain</td></tr></tbody></table>
<hr>
<h2 class="anchor anchorTargetHideOnScrollNavbar_vjPI" id="production-metrics">Production metrics<a href="https://dheerajinampudi.com/blog/spaces/ai-space/llm-evaluation-guide#production-metrics" class="hash-link" aria-label="Direct link to Production metrics" title="Direct link to Production metrics" translate="no">​</a></h2>
<h3 class="anchor anchorTargetHideOnScrollNavbar_vjPI" id="performance">Performance<a href="https://dheerajinampudi.com/blog/spaces/ai-space/llm-evaluation-guide#performance" class="hash-link" aria-label="Direct link to Performance" title="Direct link to Performance" translate="no">​</a></h3>
<table><thead><tr><th>Metric</th><th>What to measure</th><th>Why it matters</th></tr></thead><tbody><tr><td><strong>Throughput</strong></td><td>Tokens/sec, req/sec at target context length</td><td>Real production cost = throughput × latency</td></tr><tr><td><strong>Latency</strong></td><td>Time-to-first-token (TTFT) + per-token decode speed</td><td>UX + agent planning loops</td></tr><tr><td><strong>Determinism</strong></td><td>Same input → same output?</td><td>Needed for CI, debugging, compliance</td></tr></tbody></table>
<h3 class="anchor anchorTargetHideOnScrollNavbar_vjPI" id="reliability">Reliability<a href="https://dheerajinampudi.com/blog/spaces/ai-space/llm-evaluation-guide#reliability" class="hash-link" aria-label="Direct link to Reliability" title="Direct link to Reliability" translate="no">​</a></h3>
<table><thead><tr><th>Aspect</th><th>Details</th></tr></thead><tbody><tr><td><strong>Safety/refusal</strong></td><td>Guardrails (inline or post-hoc classifier); can impact valid request completion</td></tr><tr><td><strong>Context management</strong></td><td>How you place info in prompt; RAG placement; "lost in middle" mitigation</td></tr><tr><td><strong>Failure modes</strong></td><td>Known weak spots: Where does it hallucinate? Over-refuse? Loop?</td></tr></tbody></table>
<hr>
<h2 class="anchor anchorTargetHideOnScrollNavbar_vjPI" id="customization">Customization<a href="https://dheerajinampudi.com/blog/spaces/ai-space/llm-evaluation-guide#customization" class="hash-link" aria-label="Direct link to Customization" title="Direct link to Customization" translate="no">​</a></h2>
<table><thead><tr><th>Parameter</th><th>What it is</th><th>Key questions</th><th>Tradeoffs</th></tr></thead><tbody><tr><td><strong>Fine-tuning compatibility</strong></td><td>Whether you can adapt it</td><td>LoRA/QLoRA? Full finetune? SFT pipeline?</td><td>Drift, regressions, catastrophic forgetting</td></tr><tr><td><strong>Instruction tuning depth</strong></td><td>Breadth/quality of instruction-following</td><td>Multi-turn? Code-specific SFT?</td><td>Overfit to chatty style; refusals</td></tr></tbody></table>
<hr>
<h2 class="anchor anchorTargetHideOnScrollNavbar_vjPI" id="quick-evaluation-checklist">Quick evaluation checklist<a href="https://dheerajinampudi.com/blog/spaces/ai-space/llm-evaluation-guide#quick-evaluation-checklist" class="hash-link" aria-label="Direct link to Quick evaluation checklist" title="Direct link to Quick evaluation checklist" translate="no">​</a></h2>
<p>Before deploying any model, run these smell tests:</p>
<ul class="contains-task-list containsTaskList_mC6p">
<li class="task-list-item"><input type="checkbox" disabled=""> <strong>JSON/Structured output</strong> — Can it reliably emit valid JSON for 100 requests?</li>
<li class="task-list-item"><input type="checkbox" disabled=""> <strong>Tool calling</strong> — Does it call tools correctly, or does it "think" endlessly?</li>
<li class="task-list-item"><input type="checkbox" disabled=""> <strong>Long context</strong> — Does it remember constraints from 50k tokens ago?</li>
<li class="task-list-item"><input type="checkbox" disabled=""> <strong>Code patterns</strong> — Does it know modern frameworks (React 18, TypeScript 5)?</li>
<li class="task-list-item"><input type="checkbox" disabled=""> <strong>Determinism</strong> — Do tests flake? (1/20 runs fail = determinism gap)</li>
<li class="task-list-item"><input type="checkbox" disabled=""> <strong>Refusals</strong> — Does it refuse benign requests too often?</li>
</ul>
<hr>
<blockquote>
<p><strong>Remember:</strong> Predictability beats raw intelligence. A model that's slightly less capable but reliable is often better for production than a brilliant but brittle one.</p>
</blockquote>]]></content:encoded>
            <category>Spaces</category>
            <category>AI Space</category>
            <category>Production AI</category>
        </item>
        <item>
            <title><![CDATA[Stages of AI tuning]]></title>
            <link>https://dheerajinampudi.com/blog/spaces/ai-space/stages-of-ai-tuning</link>
            <guid>https://dheerajinampudi.com/blog/spaces/ai-space/stages-of-ai-tuning</guid>
            <pubDate>Sun, 08 Dec 2024 00:00:00 GMT</pubDate>
            <description><![CDATA[The four ways to customize an LLM, from prompt engineering to RAG, fine-tuning and pre-training, and how to choose between them.]]></description>
            <content:encoded><![CDATA[<p>AI customization is a spectrum. At one end is zero-cost prompt engineering; at the other is pre-training a large language model (LLM) from scratch. Each stage brings different levels of complexity, data requirements and organizational investment. Understanding the stages helps you pick the right approach for your business or research needs.</p>
<h2 class="anchor anchorTargetHideOnScrollNavbar_vjPI" id="the-four-stages">The four stages<a href="https://dheerajinampudi.com/blog/spaces/ai-space/stages-of-ai-tuning#the-four-stages" class="hash-link" aria-label="Direct link to The four stages" title="Direct link to The four stages" translate="no">​</a></h2>
<p>The main strategies for tuning or customizing an AI model are:</p>
<ol>
<li class="">
<p><strong>Prompt engineering</strong></p>
<p>At the simplest end of the spectrum lies prompt engineering. Instead of altering the model's parameters, you craft carefully worded prompts to guide its outputs. It needs no additional data or lengthy training. It's quick, cost-effective and gives immediate improvements, but offers less fine-grained control.</p>
</li>
<li class="">
<p><strong>Retrieval-augmented generation (RAG)</strong></p>
<p>RAG brings external knowledge sources, such as a vector database or an up-to-date knowledge base, into the model's responses. By retrieving relevant information at query time, the model can give more accurate and contextually rich answers without retraining. It adds complexity and inference cost, but remains a powerful way to update the model's "knowledge" on the fly.</p>
</li>
<li class="">
<p><strong>Fine-tuning</strong></p>
<p>For organizations that need more specialized control, fine-tuning is the next level. Training a pre-existing LLM on your domain-specific data refines its behavior and outputs. It requires labeled data, compute and a longer training period than prompt engineering or RAG, but it gives granular control over the model's performance and can significantly improve domain-specific accuracy.</p>
</li>
<li class="">
<p><strong>Pre-training</strong></p>
<p>The most resource-intensive end of the spectrum is pre-training an LLM from scratch on massive datasets. It provides maximum control and a tailor-made model architecture aligned with your needs. It also demands significant time (often weeks or months), computational power and large-scale curated data. It's typically pursued by large organizations or research teams aiming for pioneering innovations.</p>
</li>
</ol>
<h2 class="anchor anchorTargetHideOnScrollNavbar_vjPI" id="visualizing-the-spectrum">Visualizing the spectrum<a href="https://dheerajinampudi.com/blog/spaces/ai-space/stages-of-ai-tuning#visualizing-the-spectrum" class="hash-link" aria-label="Direct link to Visualizing the spectrum" title="Direct link to Visualizing the spectrum" translate="no">​</a></h2>
<p><img decoding="async" loading="lazy" alt="Chart plotting prompt engineering, RAG, fine-tuning and pre-training with time and complexity on the vertical axis and response quality and organizational maturity on the horizontal axis" src="https://dheerajinampudi.com/assets/images/model_tuning-c88d2af962a020c56676aa516f8ce8c4.png" width="1024" height="768" class="img_ev3q"></p>
<p>The chart above plots these stages against <strong>time/complexity</strong> and <strong>response quality/organizational maturity</strong>. As you move from prompt engineering to RAG, then to fine-tuning and finally pre-training, both the resource investment and the level of achievable specificity increase.</p>
<h2 class="anchor anchorTargetHideOnScrollNavbar_vjPI" id="detailed-comparison">Detailed comparison<a href="https://dheerajinampudi.com/blog/spaces/ai-space/stages-of-ai-tuning#detailed-comparison" class="hash-link" aria-label="Direct link to Detailed comparison" title="Direct link to Detailed comparison" translate="no">​</a></h2>
<p><img decoding="async" loading="lazy" alt="Comparison table of prompt engineering, RAG, fine-tuning and pre-training with their definitions, primary use cases, data requirements, advantages and considerations" src="https://dheerajinampudi.com/assets/images/4_steps_AI-c2a9ff61b6cecb7effbdadf8795f6ae3.jpg" width="2316" height="1080" class="img_ev3q"></p>
<p><em>Source: Databricks blog.</em></p>
<p>The table above compares the methods across these attributes:</p>
<table><thead><tr><th>Method</th><th>Data requirements</th><th>Training time</th><th>Advantages</th><th>Considerations</th></tr></thead><tbody><tr><td><strong>Prompt engineering</strong></td><td>None</td><td>None</td><td>Fast and cost-effective</td><td>Limited control</td></tr><tr><td><strong>RAG</strong></td><td>External knowledge base</td><td>Moderate (e.g., computing embeddings)</td><td>Dynamically updated context</td><td>Longer prompts and increased inference costs</td></tr><tr><td><strong>Fine-tuning</strong></td><td>Thousands of domain-specific samples</td><td>Moderate to long (depending on dataset size)</td><td>Granular domain specialization</td><td>Requires labeled data and more compute</td></tr><tr><td><strong>Pre-training</strong></td><td>Billions to trillions of tokens</td><td>Long (days to many weeks)</td><td>Maximum control and tailor-fit</td><td>Extremely resource-intensive</td></tr></tbody></table>
<h2 class="anchor anchorTargetHideOnScrollNavbar_vjPI" id="choosing-the-right-approach">Choosing the right approach<a href="https://dheerajinampudi.com/blog/spaces/ai-space/stages-of-ai-tuning#choosing-the-right-approach" class="hash-link" aria-label="Direct link to Choosing the right approach" title="Direct link to Choosing the right approach" translate="no">​</a></h2>
<p>The right stage depends on your organization's goals, resources and maturity:</p>
<table><thead><tr><th>If you need…</th><th>Start with</th></tr></thead><tbody><tr><td>Immediate results, no additional data</td><td>Prompt engineering</td></tr><tr><td>Dynamic knowledge integration</td><td>Retrieval augmentation (RAG)</td></tr><tr><td>Specific domain expertise</td><td>Fine-tuning</td></tr><tr><td>Full control and customization</td><td>Pre-training</td></tr></tbody></table>
<p>By understanding each stage's trade-offs, you can invest in the approach that best fits your constraints and aspirations.</p>]]></content:encoded>
            <category>Spaces</category>
            <category>AI Space</category>
        </item>
        <item>
            <title><![CDATA[Transformer links]]></title>
            <link>https://dheerajinampudi.com/blog/spaces/ai-space/transformer-links</link>
            <guid>https://dheerajinampudi.com/blog/spaces/ai-space/transformer-links</guid>
            <pubDate>Fri, 15 Nov 2024 00:00:00 GMT</pubDate>
            <description><![CDATA[Three interactive resources for seeing inside transformers, from Transformer Explainer and LLM Visualization to Transformer Circuits.]]></description>
            <content:encoded><![CDATA[<p>Interactive tools and references for seeing how transformer models work inside.</p>
<h2 class="anchor anchorTargetHideOnScrollNavbar_vjPI" id="transformer-explainer">Transformer Explainer<a href="https://dheerajinampudi.com/blog/spaces/ai-space/transformer-links#transformer-explainer" class="hash-link" aria-label="Direct link to Transformer Explainer" title="Direct link to Transformer Explainer" translate="no">​</a></h2>
<p>An interactive tool for visualizing and understanding the inner workings of transformer models. It shows how these models process and generate text across layers and attention heads.</p>
<p><a href="https://poloclub.github.io/transformer-explainer/" target="_blank" rel="noopener noreferrer" class="">Open Transformer Explainer</a></p>
<h2 class="anchor anchorTargetHideOnScrollNavbar_vjPI" id="llm-visualization">LLM Visualization<a href="https://dheerajinampudi.com/blog/spaces/ai-space/transformer-links#llm-visualization" class="hash-link" aria-label="Direct link to LLM Visualization" title="Direct link to LLM Visualization" translate="no">​</a></h2>
<p>A visualization platform with detailed, intuitive explorations of large language models (LLMs). It helps you build a deeper understanding of their structure, parameters and behavior.</p>
<p><a href="https://bbycroft.net/llm" target="_blank" rel="noopener noreferrer" class="">Open LLM Visualization</a></p>
<h2 class="anchor anchorTargetHideOnScrollNavbar_vjPI" id="transformer-circuits">Transformer Circuits<a href="https://dheerajinampudi.com/blog/spaces/ai-space/transformer-links#transformer-circuits" class="hash-link" aria-label="Direct link to Transformer Circuits" title="Direct link to Transformer Circuits" translate="no">​</a></h2>
<p>A comprehensive resource on the circuit-level functioning of transformer models. It focuses on frameworks and simplifications that demystify their decision-making processes and internal representations.</p>
<p><a href="https://transformer-circuits.pub/2021/framework/index.html#model-simplifications" target="_blank" rel="noopener noreferrer" class="">Open A Mathematical Framework for Transformer Circuits</a></p>]]></content:encoded>
            <category>Spaces</category>
            <category>AI Space</category>
            <category>Transformers</category>
        </item>
        <item>
            <title><![CDATA[Understanding attention and transformer architectures]]></title>
            <link>https://dheerajinampudi.com/blog/spaces/ai-space/attention-and-transformers</link>
            <guid>https://dheerajinampudi.com/blog/spaces/ai-space/attention-and-transformers</guid>
            <pubDate>Thu, 10 Oct 2024 00:00:00 GMT</pubDate>
            <description><![CDATA[Attention from first principles (Q/K/V, self-attention, multi-head, positional encoding) up to transformer encoders and decoders.]]></description>
            <content:encoded><![CDATA[<h2 class="anchor anchorTargetHideOnScrollNavbar_vjPI" id="three-mental-models">Three mental models<a href="https://dheerajinampudi.com/blog/spaces/ai-space/attention-and-transformers#three-mental-models" class="hash-link" aria-label="Direct link to Three mental models" title="Direct link to Three mental models" translate="no">​</a></h2>
<ul>
<li class=""><strong>RNNs (Storyteller):</strong> Reads a story word by word, remembering only what came before to predict what comes next.</li>
<li class=""><strong>CNNs (Photographer):</strong> Focuses on details in patches of an image and combines them to form a complete picture.</li>
<li class=""><strong>Transformers (Collaborator):</strong> Reads the entire story at once and learns relationships between all parts simultaneously, identifying which details to focus on globally.</li>
</ul>
<p>Let's break this into two key parts: the <strong>attention mechanism</strong> and the <strong>transformer architecture</strong>. The architecture builds on attention, so mastering attention first is critical to understanding the full transformer model.</p>
<h2 class="anchor anchorTargetHideOnScrollNavbar_vjPI" id="1-attention-mechanism">1. Attention mechanism<a href="https://dheerajinampudi.com/blog/spaces/ai-space/attention-and-transformers#1-attention-mechanism" class="hash-link" aria-label="Direct link to 1. Attention mechanism" title="Direct link to 1. Attention mechanism" translate="no">​</a></h2>
<p>The <strong>attention mechanism</strong> is the foundation of Transformers. It allows the model to <strong>focus on relevant parts of the input sequence</strong> when generating outputs, addressing limitations of RNNs (like long-range dependencies).</p>
<h3 class="anchor anchorTargetHideOnScrollNavbar_vjPI" id="core-idea">Core idea<a href="https://dheerajinampudi.com/blog/spaces/ai-space/attention-and-transformers#core-idea" class="hash-link" aria-label="Direct link to Core idea" title="Direct link to Core idea" translate="no">​</a></h3>
<ul>
<li class="">Instead of processing sequences step by step (like RNNs), attention compares <strong>all tokens</strong> in the input to each other to determine relevance.</li>
<li class="">It assigns <strong>weights (importance scores)</strong> to each token based on how much they contribute to understanding a given token or query.</li>
</ul>
<h3 class="anchor anchorTargetHideOnScrollNavbar_vjPI" id="key-components-of-attention">Key components of attention<a href="https://dheerajinampudi.com/blog/spaces/ai-space/attention-and-transformers#key-components-of-attention" class="hash-link" aria-label="Direct link to Key components of attention" title="Direct link to Key components of attention" translate="no">​</a></h3>
<p>Attention is built around the <strong>Query (Q)</strong>, <strong>Key (K)</strong>, and <strong>Value (V)</strong> paradigm.</p>
<ol>
<li class="">
<p><strong>Query (Q):</strong></p>
<ul>
<li class="">Represents what you're looking for.</li>
<li class="">Example: In the sentence "The cat sat on the mat," if we're focusing on "cat," the query is related to "cat."</li>
</ul>
</li>
<li class="">
<p><strong>Key (K):</strong></p>
<ul>
<li class="">Represents the characteristics of each token.</li>
<li class="">Example: Every word in the sequence has a key, encoding its meaning.</li>
</ul>
</li>
<li class="">
<p><strong>Value (V):</strong></p>
<ul>
<li class="">Represents the actual content of the tokens.</li>
<li class="">Example: "The", "cat", "sat", etc., are passed as values.</li>
</ul>
</li>
</ol>
<h3 class="anchor anchorTargetHideOnScrollNavbar_vjPI" id="self-attention-formula">Self-attention formula<a href="https://dheerajinampudi.com/blog/spaces/ai-space/attention-and-transformers#self-attention-formula" class="hash-link" aria-label="Direct link to Self-attention formula" title="Direct link to Self-attention formula" translate="no">​</a></h3>
<p>Attention computes relevance scores for each token pair using dot products:</p>
<span class="katex-display"><span class="katex"><span class="katex-mathml"><math xmlns="http://www.w3.org/1998/Math/MathML" display="block"><semantics><mrow><mtext>Attention(Q,&nbsp;K,&nbsp;V)</mtext><mo>=</mo><mtext>softmax</mtext><mrow><mo fence="true">(</mo><mfrac><mrow><mi>Q</mi><msup><mi>K</mi><mi>T</mi></msup></mrow><msqrt><msub><mi>d</mi><mi>k</mi></msub></msqrt></mfrac><mo fence="true">)</mo></mrow><mi>V</mi></mrow><annotation encoding="application/x-tex">\text{Attention(Q, K, V)} = \text{softmax}\left(\frac{QK^T}{\sqrt{d_k}}\right)V</annotation></semantics></math></span><span class="katex-html" aria-hidden="true"><span class="katex-base"><span class="katex-strut" style="height:1em;vertical-align:-0.25em"></span><span class="mord text"><span class="mord">Attention(Q,&nbsp;K,&nbsp;V)</span></span><span class="mspace" style="margin-right:0.2778em"></span><span class="mrel">=</span><span class="mspace" style="margin-right:0.2778em"></span></span><span class="katex-base"><span class="katex-strut" style="height:2.4684em;vertical-align:-0.95em"></span><span class="mord text"><span class="mord">softmax</span></span><span class="mspace" style="margin-right:0.1667em"></span><span class="minner"><span class="mopen delimcenter" style="top:0em"><span class="delimsizing size3">(</span></span><span class="mord"><span class="mopen nulldelimiter"></span><span class="mfrac"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height:1.5183em"><span style="top:-2.2528em"><span class="pstrut" style="height:3em"></span><span class="mord"><span class="mord sqrt"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height:0.8572em"><span class="svg-align" style="top:-3em"><span class="pstrut" style="height:3em"></span><span class="mord" style="padding-left:0.833em"><span class="mord"><span class="mord mathnormal">d</span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height:0.3361em"><span style="top:-2.55em;margin-left:0em;margin-right:0.05em"><span class="pstrut" style="height:2.7em"></span><span class="katex-sizing reset-size6 size3 mtight"><span class="mord mathnormal mtight" style="margin-right:0.0315em">k</span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height:0.15em"><span></span></span></span></span></span></span></span></span><span style="top:-2.8172em"><span class="pstrut" style="height:3em"></span><span class="hide-tail" style="min-width:0.853em;height:1.08em"><svg xmlns="http://www.w3.org/2000/svg" width="400em" height="1.08em" viewBox="0 0 400000 1080" preserveAspectRatio="xMinYMin slice"><path d="M95,702
c-2.7,0,-7.17,-2.7,-13.5,-8c-5.8,-5.3,-9.5,-10,-9.5,-14
c0,-2,0.3,-3.3,1,-4c1.3,-2.7,23.83,-20.7,67.5,-54
c44.2,-33.3,65.8,-50.3,66.5,-51c1.3,-1.3,3,-2,5,-2c4.7,0,8.7,3.3,12,10
s173,378,173,378c0.7,0,35.3,-71,104,-213c68.7,-142,137.5,-285,206.5,-429
c69,-144,104.5,-217.7,106.5,-221
l0 -0
c5.3,-9.3,12,-14,20,-14
H400000v40H845.2724
s-225.272,467,-225.272,467s-235,486,-235,486c-2.7,4.7,-9,7,-19,7
c-6,0,-10,-1,-12,-3s-194,-422,-194,-422s-65,47,-65,47z
M834 80h400000v40h-400000z"></path></svg></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height:0.1828em"><span></span></span></span></span></span></span></span><span style="top:-3.23em"><span class="pstrut" style="height:3em"></span><span class="frac-line" style="border-bottom-width:0.04em"></span></span><span style="top:-3.677em"><span class="pstrut" style="height:3em"></span><span class="mord"><span class="mord mathnormal">Q</span><span class="mord"><span class="mord mathnormal" style="margin-right:0.0715em">K</span><span class="msupsub"><span class="vlist-t"><span class="vlist-r"><span class="vlist" style="height:0.8413em"><span style="top:-3.063em;margin-right:0.05em"><span class="pstrut" style="height:2.7em"></span><span class="katex-sizing reset-size6 size3 mtight"><span class="mord mathnormal mtight" style="margin-right:0.1389em">T</span></span></span></span></span></span></span></span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height:0.93em"><span></span></span></span></span></span><span class="mclose nulldelimiter"></span></span><span class="mclose delimcenter" style="top:0em"><span class="delimsizing size3">)</span></span></span><span class="mspace" style="margin-right:0.1667em"></span><span class="mord mathnormal" style="margin-right:0.2222em">V</span></span></span></span></span>
<ul>
<li class=""><strong>QKᵀ:</strong> Measures similarity between query and key.</li>
<li class=""><strong>dₖ:</strong> Scaling factor to avoid large dot products in high dimensions.</li>
<li class=""><strong>softmax:</strong> Converts scores into probabilities (attention weights).</li>
<li class=""><strong>V:</strong> Final weighted sum of values, representing the focused context.</li>
</ul>
<h3 class="anchor anchorTargetHideOnScrollNavbar_vjPI" id="example-of-self-attention">Example of self-attention<a href="https://dheerajinampudi.com/blog/spaces/ai-space/attention-and-transformers#example-of-self-attention" class="hash-link" aria-label="Direct link to Example of self-attention" title="Direct link to Example of self-attention" translate="no">​</a></h3>
<p>Input sentence: <code>"The cat sat on the mat."</code></p>
<p>If the focus is on "cat":</p>
<ul>
<li class="">Query (<code>Q</code>) = "cat".</li>
<li class="">Each word in the sentence has a <strong>Key (<code>K</code>)</strong> and <strong>Value (<code>V</code>)</strong>.</li>
<li class="">The attention mechanism calculates how much each word contributes to "cat" using QKᵀ:<!-- -->
<ul>
<li class="">"The" → Low weight (irrelevant).</li>
<li class="">"sat" → High weight (verb related to "cat").</li>
<li class="">"mat" → Low weight (less connected).</li>
</ul>
</li>
</ul>
<h2 class="anchor anchorTargetHideOnScrollNavbar_vjPI" id="2-multi-head-attention">2. Multi-head attention<a href="https://dheerajinampudi.com/blog/spaces/ai-space/attention-and-transformers#2-multi-head-attention" class="hash-link" aria-label="Direct link to 2. Multi-head attention" title="Direct link to 2. Multi-head attention" translate="no">​</a></h2>
<p>A single attention mechanism may miss nuanced relationships (e.g., syntax vs. semantics). <strong>Multi-Head Attention</strong> solves this by:</p>
<ul>
<li class="">Running multiple attention mechanisms (heads) in parallel.</li>
<li class="">Each head captures different relationships (e.g., word order, context, synonyms).</li>
<li class="">Outputs are concatenated and projected into a final representation.</li>
</ul>
<span class="katex-display"><span class="katex"><span class="katex-mathml"><math xmlns="http://www.w3.org/1998/Math/MathML" display="block"><semantics><mrow><mtext>MultiHead(Q,&nbsp;K,&nbsp;V)</mtext><mo>=</mo><mtext>Concat</mtext><mo stretchy="false">(</mo><msub><mtext>head</mtext><mn>1</mn></msub><mo separator="true">,</mo><msub><mtext>head</mtext><mn>2</mn></msub><mo separator="true">,</mo><mo>…</mo><mo separator="true">,</mo><msub><mtext>head</mtext><mi>h</mi></msub><mo stretchy="false">)</mo><msup><mi>W</mi><mi>O</mi></msup></mrow><annotation encoding="application/x-tex">\text{MultiHead(Q, K, V)} = \text{Concat}(\text{head}_1, \text{head}_2, \dots, \text{head}_h)W^O</annotation></semantics></math></span><span class="katex-html" aria-hidden="true"><span class="katex-base"><span class="katex-strut" style="height:1em;vertical-align:-0.25em"></span><span class="mord text"><span class="mord">MultiHead(Q,&nbsp;K,&nbsp;V)</span></span><span class="mspace" style="margin-right:0.2778em"></span><span class="mrel">=</span><span class="mspace" style="margin-right:0.2778em"></span></span><span class="katex-base"><span class="katex-strut" style="height:1.1413em;vertical-align:-0.25em"></span><span class="mord text"><span class="mord">Concat</span></span><span class="mopen">(</span><span class="mord"><span class="mord text"><span class="mord">head</span></span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height:0.3011em"><span style="top:-2.55em;margin-right:0.05em"><span class="pstrut" style="height:2.7em"></span><span class="katex-sizing reset-size6 size3 mtight"><span class="mord mtight">1</span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height:0.15em"><span></span></span></span></span></span></span><span class="mpunct">,</span><span class="mspace" style="margin-right:0.1667em"></span><span class="mord"><span class="mord text"><span class="mord">head</span></span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height:0.3011em"><span style="top:-2.55em;margin-right:0.05em"><span class="pstrut" style="height:2.7em"></span><span class="katex-sizing reset-size6 size3 mtight"><span class="mord mtight">2</span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height:0.15em"><span></span></span></span></span></span></span><span class="mpunct">,</span><span class="mspace" style="margin-right:0.1667em"></span><span class="minner">…</span><span class="mspace" style="margin-right:0.1667em"></span><span class="mpunct">,</span><span class="mspace" style="margin-right:0.1667em"></span><span class="mord"><span class="mord text"><span class="mord">head</span></span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height:0.3361em"><span style="top:-2.55em;margin-right:0.05em"><span class="pstrut" style="height:2.7em"></span><span class="katex-sizing reset-size6 size3 mtight"><span class="mord mathnormal mtight">h</span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height:0.15em"><span></span></span></span></span></span></span><span class="mclose">)</span><span class="mord"><span class="mord mathnormal" style="margin-right:0.1389em">W</span><span class="msupsub"><span class="vlist-t"><span class="vlist-r"><span class="vlist" style="height:0.8913em"><span style="top:-3.113em;margin-right:0.05em"><span class="pstrut" style="height:2.7em"></span><span class="katex-sizing reset-size6 size3 mtight"><span class="mord mathnormal mtight" style="margin-right:0.0278em">O</span></span></span></span></span></span></span></span></span></span></span></span>
<h2 class="anchor anchorTargetHideOnScrollNavbar_vjPI" id="3-positional-encoding">3. Positional encoding<a href="https://dheerajinampudi.com/blog/spaces/ai-space/attention-and-transformers#3-positional-encoding" class="hash-link" aria-label="Direct link to 3. Positional encoding" title="Direct link to 3. Positional encoding" translate="no">​</a></h2>
<p>Transformers process sequences in parallel, so they lack inherent order information (unlike RNNs). <strong>Positional Encoding</strong> adds order back:</p>
<ul>
<li class="">Each token embedding is enriched with a positional vector.</li>
<li class="">Common approach: Use sine and cosine functions for each position.</li>
</ul>
<span class="katex-display"><span class="katex"><span class="katex-mathml"><math xmlns="http://www.w3.org/1998/Math/MathML" display="block"><semantics><mrow><mtext>PosEnc(pos,&nbsp;2i)</mtext><mo>=</mo><mi>sin</mi><mo>⁡</mo><mo stretchy="false">(</mo><mfrac><mtext>pos</mtext><msup><mn>10000</mn><mrow><mn>2</mn><mi>i</mi><mi mathvariant="normal">/</mi><mi>d</mi></mrow></msup></mfrac><mo stretchy="false">)</mo><mo separator="true">,</mo><mspace width="1em"></mspace><mtext>PosEnc(pos,&nbsp;2i+1)</mtext><mo>=</mo><mi>cos</mi><mo>⁡</mo><mo stretchy="false">(</mo><mfrac><mtext>pos</mtext><msup><mn>10000</mn><mrow><mn>2</mn><mi>i</mi><mi mathvariant="normal">/</mi><mi>d</mi></mrow></msup></mfrac><mo stretchy="false">)</mo></mrow><annotation encoding="application/x-tex">\text{PosEnc(pos, 2i)} = \sin(\frac{\text{pos}}{10000^{2i/d}}), \quad \text{PosEnc(pos, 2i+1)} = \cos(\frac{\text{pos}}{10000^{2i/d}})</annotation></semantics></math></span><span class="katex-html" aria-hidden="true"><span class="katex-base"><span class="katex-strut" style="height:1em;vertical-align:-0.25em"></span><span class="mord text"><span class="mord">PosEnc(pos,&nbsp;2i)</span></span><span class="mspace" style="margin-right:0.2778em"></span><span class="mrel">=</span><span class="mspace" style="margin-right:0.2778em"></span></span><span class="katex-base"><span class="katex-strut" style="height:1.8116em;vertical-align:-0.704em"></span><span class="mop">sin</span><span class="mopen">(</span><span class="mord"><span class="mopen nulldelimiter"></span><span class="mfrac"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height:1.1076em"><span style="top:-2.296em"><span class="pstrut" style="height:3em"></span><span class="mord"><span class="mord">1000</span><span class="mord"><span class="mord">0</span><span class="msupsub"><span class="vlist-t"><span class="vlist-r"><span class="vlist" style="height:0.814em"><span style="top:-2.989em;margin-right:0.05em"><span class="pstrut" style="height:2.7em"></span><span class="katex-sizing reset-size6 size3 mtight"><span class="mord mtight"><span class="mord mtight">2</span><span class="mord mathnormal mtight">i</span><span class="mord mtight">/</span><span class="mord mathnormal mtight">d</span></span></span></span></span></span></span></span></span></span></span><span style="top:-3.23em"><span class="pstrut" style="height:3em"></span><span class="frac-line" style="border-bottom-width:0.04em"></span></span><span style="top:-3.677em"><span class="pstrut" style="height:3em"></span><span class="mord"><span class="mord text"><span class="mord">pos</span></span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height:0.704em"><span></span></span></span></span></span><span class="mclose nulldelimiter"></span></span><span class="mclose">)</span><span class="mpunct">,</span><span class="mspace" style="margin-right:1em"></span><span class="mspace" style="margin-right:0.1667em"></span><span class="mord text"><span class="mord">PosEnc(pos,&nbsp;2i+1)</span></span><span class="mspace" style="margin-right:0.2778em"></span><span class="mrel">=</span><span class="mspace" style="margin-right:0.2778em"></span></span><span class="katex-base"><span class="katex-strut" style="height:1.8116em;vertical-align:-0.704em"></span><span class="mop">cos</span><span class="mopen">(</span><span class="mord"><span class="mopen nulldelimiter"></span><span class="mfrac"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height:1.1076em"><span style="top:-2.296em"><span class="pstrut" style="height:3em"></span><span class="mord"><span class="mord">1000</span><span class="mord"><span class="mord">0</span><span class="msupsub"><span class="vlist-t"><span class="vlist-r"><span class="vlist" style="height:0.814em"><span style="top:-2.989em;margin-right:0.05em"><span class="pstrut" style="height:2.7em"></span><span class="katex-sizing reset-size6 size3 mtight"><span class="mord mtight"><span class="mord mtight">2</span><span class="mord mathnormal mtight">i</span><span class="mord mtight">/</span><span class="mord mathnormal mtight">d</span></span></span></span></span></span></span></span></span></span></span><span style="top:-3.23em"><span class="pstrut" style="height:3em"></span><span class="frac-line" style="border-bottom-width:0.04em"></span></span><span style="top:-3.677em"><span class="pstrut" style="height:3em"></span><span class="mord"><span class="mord text"><span class="mord">pos</span></span></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height:0.704em"><span></span></span></span></span></span><span class="mclose nulldelimiter"></span></span><span class="mclose">)</span></span></span></span></span>
<h2 class="anchor anchorTargetHideOnScrollNavbar_vjPI" id="4-transformer-architecture">4. Transformer architecture<a href="https://dheerajinampudi.com/blog/spaces/ai-space/attention-and-transformers#4-transformer-architecture" class="hash-link" aria-label="Direct link to 4. Transformer architecture" title="Direct link to 4. Transformer architecture" translate="no">​</a></h2>
<p>The <strong>Transformer</strong> builds on attention by combining <strong>Self-Attention</strong>, <strong>Feed-Forward Networks (FFN)</strong>, and <strong>Residual Connections</strong> in a modular, stackable architecture.</p>
<h3 class="anchor anchorTargetHideOnScrollNavbar_vjPI" id="transformer-encoder">Transformer encoder<a href="https://dheerajinampudi.com/blog/spaces/ai-space/attention-and-transformers#transformer-encoder" class="hash-link" aria-label="Direct link to Transformer encoder" title="Direct link to Transformer encoder" translate="no">​</a></h3>
<p>Encodes the input into a rich contextual representation.</p>
<p><strong>One encoder block:</strong></p>
<ol>
<li class="">
<p><strong>Multi-head self-attention:</strong></p>
<ul>
<li class="">Computes relationships between all tokens in the input.</li>
<li class="">Outputs context-aware embeddings for each token.</li>
</ul>
</li>
<li class="">
<p><strong>Feed-forward network (FFN):</strong></p>
<ul>
<li class="">Applies two dense layers with non-linear activation.</li>
<li class="">Refines the token representations.</li>
</ul>
</li>
<li class="">
<p><strong>Residual connections and layer normalization:</strong></p>
<ul>
<li class="">Residual connections stabilize training by adding the input back to the output of attention/FFN layers.</li>
<li class="">Layer normalization improves gradient flow.</li>
</ul>
</li>
</ol>
<p><strong>Structure:</strong></p>
<span class="katex-display"><span class="katex"><span class="katex-mathml"><math xmlns="http://www.w3.org/1998/Math/MathML" display="block"><semantics><mrow><mtext>Input&nbsp;Embeddings</mtext><mover><mo stretchy="true" minsize="3.0em">→</mo><mpadded width="+0.6em" lspace="0.3em"><mtext>Self-Attention</mtext></mpadded></mover><mover><mo stretchy="true" minsize="3.0em">→</mo><mpadded width="+0.6em" lspace="0.3em"><mtext>FFN</mtext></mpadded></mover><mover><mo stretchy="true" minsize="3.0em">→</mo><mpadded width="+0.6em" lspace="0.3em"><mtext>Normalization</mtext></mpadded></mover><mtext>Output&nbsp;Embeddings</mtext></mrow><annotation encoding="application/x-tex">\text{Input Embeddings} \xrightarrow{\text{Self-Attention}} \xrightarrow{\text{FFN}} \xrightarrow{\text{Normalization}} \text{Output Embeddings}</annotation></semantics></math></span><span class="katex-html" aria-hidden="true"><span class="katex-base"><span class="katex-strut" style="height:1.3025em;vertical-align:-0.1944em"></span><span class="mord text"><span class="mord">Input&nbsp;Embeddings</span></span><span class="mspace" style="margin-right:0.2778em"></span><span class="mrel x-arrow"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height:1.1081em"><span style="top:-3.322em"><span class="pstrut" style="height:2.7em"></span><span class="katex-sizing reset-size6 size3 mtight x-arrow-pad"><span class="mord mtight"><span class="mord text mtight"><span class="mord mtight">Self-Attention</span></span></span></span></span><span class="svg-align" style="top:-2.689em"><span class="pstrut" style="height:2.7em"></span><span class="hide-tail" style="height:0.522em;min-width:1.469em"><svg xmlns="http://www.w3.org/2000/svg" width="400em" height="0.522em" viewBox="0 0 400000 522" preserveAspectRatio="xMaxYMin slice"><path d="M0 241v40h399891c-47.3 35.3-84 78-110 128
-16.7 32-27.7 63.7-33 95 0 1.3-.2 2.7-.5 4-.3 1.3-.5 2.3-.5 3 0 7.3 6.7 11 20
 11 8 0 13.2-.8 15.5-2.5 2.3-1.7 4.2-5.5 5.5-11.5 2-13.3 5.7-27 11-41 14.7-44.7
 39-84.5 73-119.5s73.7-60.2 119-75.5c6-2 9-5.7 9-11s-3-9-9-11c-45.3-15.3-85
-40.5-119-75.5s-58.3-74.8-73-119.5c-4.7-14-8.3-27.3-11-40-1.3-6.7-3.2-10.8-5.5
-12.5-2.3-1.7-7.5-2.5-15.5-2.5-14 0-21 3.7-21 11 0 2 2 10.3 6 25 20.7 83.3 67
 151.7 139 205zm0 0v40h399900v-40z"></path></svg></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height:0.011em"><span></span></span></span></span></span></span><span class="katex-base"><span class="katex-strut" style="height:1.1113em;vertical-align:-0.011em"></span><span class="mrel x-arrow"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height:1.1003em"><span style="top:-3.322em"><span class="pstrut" style="height:2.7em"></span><span class="katex-sizing reset-size6 size3 mtight x-arrow-pad"><span class="mord mtight"><span class="mord text mtight"><span class="mord mtight">FFN</span></span></span></span></span><span class="svg-align" style="top:-2.689em"><span class="pstrut" style="height:2.7em"></span><span class="hide-tail" style="height:0.522em;min-width:1.469em"><svg xmlns="http://www.w3.org/2000/svg" width="400em" height="0.522em" viewBox="0 0 400000 522" preserveAspectRatio="xMaxYMin slice"><path d="M0 241v40h399891c-47.3 35.3-84 78-110 128
-16.7 32-27.7 63.7-33 95 0 1.3-.2 2.7-.5 4-.3 1.3-.5 2.3-.5 3 0 7.3 6.7 11 20
 11 8 0 13.2-.8 15.5-2.5 2.3-1.7 4.2-5.5 5.5-11.5 2-13.3 5.7-27 11-41 14.7-44.7
 39-84.5 73-119.5s73.7-60.2 119-75.5c6-2 9-5.7 9-11s-3-9-9-11c-45.3-15.3-85
-40.5-119-75.5s-58.3-74.8-73-119.5c-4.7-14-8.3-27.3-11-40-1.3-6.7-3.2-10.8-5.5
-12.5-2.3-1.7-7.5-2.5-15.5-2.5-14 0-21 3.7-21 11 0 2 2 10.3 6 25 20.7 83.3 67
 151.7 139 205zm0 0v40h399900v-40z"></path></svg></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height:0.011em"><span></span></span></span></span></span></span><span class="katex-base"><span class="katex-strut" style="height:1.1191em;vertical-align:-0.011em"></span><span class="mrel x-arrow"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height:1.1081em"><span style="top:-3.322em"><span class="pstrut" style="height:2.7em"></span><span class="katex-sizing reset-size6 size3 mtight x-arrow-pad"><span class="mord mtight"><span class="mord text mtight"><span class="mord mtight">Normalization</span></span></span></span></span><span class="svg-align" style="top:-2.689em"><span class="pstrut" style="height:2.7em"></span><span class="hide-tail" style="height:0.522em;min-width:1.469em"><svg xmlns="http://www.w3.org/2000/svg" width="400em" height="0.522em" viewBox="0 0 400000 522" preserveAspectRatio="xMaxYMin slice"><path d="M0 241v40h399891c-47.3 35.3-84 78-110 128
-16.7 32-27.7 63.7-33 95 0 1.3-.2 2.7-.5 4-.3 1.3-.5 2.3-.5 3 0 7.3 6.7 11 20
 11 8 0 13.2-.8 15.5-2.5 2.3-1.7 4.2-5.5 5.5-11.5 2-13.3 5.7-27 11-41 14.7-44.7
 39-84.5 73-119.5s73.7-60.2 119-75.5c6-2 9-5.7 9-11s-3-9-9-11c-45.3-15.3-85
-40.5-119-75.5s-58.3-74.8-73-119.5c-4.7-14-8.3-27.3-11-40-1.3-6.7-3.2-10.8-5.5
-12.5-2.3-1.7-7.5-2.5-15.5-2.5-14 0-21 3.7-21 11 0 2 2 10.3 6 25 20.7 83.3 67
 151.7 139 205zm0 0v40h399900v-40z"></path></svg></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height:0.011em"><span></span></span></span></span></span><span class="mspace" style="margin-right:0.2778em"></span></span><span class="katex-base"><span class="katex-strut" style="height:0.8889em;vertical-align:-0.1944em"></span><span class="mord text"><span class="mord">Output&nbsp;Embeddings</span></span></span></span></span></span>
<h3 class="anchor anchorTargetHideOnScrollNavbar_vjPI" id="transformer-decoder">Transformer decoder<a href="https://dheerajinampudi.com/blog/spaces/ai-space/attention-and-transformers#transformer-decoder" class="hash-link" aria-label="Direct link to Transformer decoder" title="Direct link to Transformer decoder" translate="no">​</a></h3>
<p>Generates the output sequence step-by-step.</p>
<p><strong>One decoder block:</strong></p>
<ol>
<li class="">
<p><strong>Masked multi-head attention:</strong></p>
<ul>
<li class="">Ensures the decoder only attends to already-generated tokens (causal attention).</li>
</ul>
</li>
<li class="">
<p><strong>Cross-attention:</strong></p>
<ul>
<li class="">Allows the decoder to attend to the encoder's outputs for context (important for tasks like translation).</li>
</ul>
</li>
<li class="">
<p><strong>Feed-forward network and residual connections:</strong></p>
<ul>
<li class="">Similar to the encoder.</li>
</ul>
</li>
</ol>
<p><strong>Structure:</strong></p>
<span class="katex-display"><span class="katex"><span class="katex-mathml"><math xmlns="http://www.w3.org/1998/Math/MathML" display="block"><semantics><mrow><mtext>Generated&nbsp;Embeddings</mtext><mover><mo stretchy="true" minsize="3.0em">→</mo><mpadded width="+0.6em" lspace="0.3em"><mtext>Masked&nbsp;Self-Attention</mtext></mpadded></mover><mover><mo stretchy="true" minsize="3.0em">→</mo><mpadded width="+0.6em" lspace="0.3em"><mtext>Cross-Attention</mtext></mpadded></mover><mover><mo stretchy="true" minsize="3.0em">→</mo><mpadded width="+0.6em" lspace="0.3em"><mtext>FFN</mtext></mpadded></mover><mtext>Next&nbsp;Token&nbsp;Embedding</mtext></mrow><annotation encoding="application/x-tex">\text{Generated Embeddings} \xrightarrow{\text{Masked Self-Attention}} \xrightarrow{\text{Cross-Attention}} \xrightarrow{\text{FFN}} \text{Next Token Embedding}</annotation></semantics></math></span><span class="katex-html" aria-hidden="true"><span class="katex-base"><span class="katex-strut" style="height:1.3025em;vertical-align:-0.1944em"></span><span class="mord text"><span class="mord">Generated&nbsp;Embeddings</span></span><span class="mspace" style="margin-right:0.2778em"></span><span class="mrel x-arrow"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height:1.1081em"><span style="top:-3.322em"><span class="pstrut" style="height:2.7em"></span><span class="katex-sizing reset-size6 size3 mtight x-arrow-pad"><span class="mord mtight"><span class="mord text mtight"><span class="mord mtight">Masked&nbsp;Self-Attention</span></span></span></span></span><span class="svg-align" style="top:-2.689em"><span class="pstrut" style="height:2.7em"></span><span class="hide-tail" style="height:0.522em;min-width:1.469em"><svg xmlns="http://www.w3.org/2000/svg" width="400em" height="0.522em" viewBox="0 0 400000 522" preserveAspectRatio="xMaxYMin slice"><path d="M0 241v40h399891c-47.3 35.3-84 78-110 128
-16.7 32-27.7 63.7-33 95 0 1.3-.2 2.7-.5 4-.3 1.3-.5 2.3-.5 3 0 7.3 6.7 11 20
 11 8 0 13.2-.8 15.5-2.5 2.3-1.7 4.2-5.5 5.5-11.5 2-13.3 5.7-27 11-41 14.7-44.7
 39-84.5 73-119.5s73.7-60.2 119-75.5c6-2 9-5.7 9-11s-3-9-9-11c-45.3-15.3-85
-40.5-119-75.5s-58.3-74.8-73-119.5c-4.7-14-8.3-27.3-11-40-1.3-6.7-3.2-10.8-5.5
-12.5-2.3-1.7-7.5-2.5-15.5-2.5-14 0-21 3.7-21 11 0 2 2 10.3 6 25 20.7 83.3 67
 151.7 139 205zm0 0v40h399900v-40z"></path></svg></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height:0.011em"><span></span></span></span></span></span></span><span class="katex-base"><span class="katex-strut" style="height:1.1113em;vertical-align:-0.011em"></span><span class="mrel x-arrow"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height:1.1003em"><span style="top:-3.322em"><span class="pstrut" style="height:2.7em"></span><span class="katex-sizing reset-size6 size3 mtight x-arrow-pad"><span class="mord mtight"><span class="mord text mtight"><span class="mord mtight">Cross-Attention</span></span></span></span></span><span class="svg-align" style="top:-2.689em"><span class="pstrut" style="height:2.7em"></span><span class="hide-tail" style="height:0.522em;min-width:1.469em"><svg xmlns="http://www.w3.org/2000/svg" width="400em" height="0.522em" viewBox="0 0 400000 522" preserveAspectRatio="xMaxYMin slice"><path d="M0 241v40h399891c-47.3 35.3-84 78-110 128
-16.7 32-27.7 63.7-33 95 0 1.3-.2 2.7-.5 4-.3 1.3-.5 2.3-.5 3 0 7.3 6.7 11 20
 11 8 0 13.2-.8 15.5-2.5 2.3-1.7 4.2-5.5 5.5-11.5 2-13.3 5.7-27 11-41 14.7-44.7
 39-84.5 73-119.5s73.7-60.2 119-75.5c6-2 9-5.7 9-11s-3-9-9-11c-45.3-15.3-85
-40.5-119-75.5s-58.3-74.8-73-119.5c-4.7-14-8.3-27.3-11-40-1.3-6.7-3.2-10.8-5.5
-12.5-2.3-1.7-7.5-2.5-15.5-2.5-14 0-21 3.7-21 11 0 2 2 10.3 6 25 20.7 83.3 67
 151.7 139 205zm0 0v40h399900v-40z"></path></svg></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height:0.011em"><span></span></span></span></span></span></span><span class="katex-base"><span class="katex-strut" style="height:1.1113em;vertical-align:-0.011em"></span><span class="mrel x-arrow"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist" style="height:1.1003em"><span style="top:-3.322em"><span class="pstrut" style="height:2.7em"></span><span class="katex-sizing reset-size6 size3 mtight x-arrow-pad"><span class="mord mtight"><span class="mord text mtight"><span class="mord mtight">FFN</span></span></span></span></span><span class="svg-align" style="top:-2.689em"><span class="pstrut" style="height:2.7em"></span><span class="hide-tail" style="height:0.522em;min-width:1.469em"><svg xmlns="http://www.w3.org/2000/svg" width="400em" height="0.522em" viewBox="0 0 400000 522" preserveAspectRatio="xMaxYMin slice"><path d="M0 241v40h399891c-47.3 35.3-84 78-110 128
-16.7 32-27.7 63.7-33 95 0 1.3-.2 2.7-.5 4-.3 1.3-.5 2.3-.5 3 0 7.3 6.7 11 20
 11 8 0 13.2-.8 15.5-2.5 2.3-1.7 4.2-5.5 5.5-11.5 2-13.3 5.7-27 11-41 14.7-44.7
 39-84.5 73-119.5s73.7-60.2 119-75.5c6-2 9-5.7 9-11s-3-9-9-11c-45.3-15.3-85
-40.5-119-75.5s-58.3-74.8-73-119.5c-4.7-14-8.3-27.3-11-40-1.3-6.7-3.2-10.8-5.5
-12.5-2.3-1.7-7.5-2.5-15.5-2.5-14 0-21 3.7-21 11 0 2 2 10.3 6 25 20.7 83.3 67
 151.7 139 205zm0 0v40h399900v-40z"></path></svg></span></span></span><span class="vlist-s">​</span></span><span class="vlist-r"><span class="vlist" style="height:0.011em"><span></span></span></span></span></span><span class="mspace" style="margin-right:0.2778em"></span></span><span class="katex-base"><span class="katex-strut" style="height:0.8889em;vertical-align:-0.1944em"></span><span class="mord text"><span class="mord">Next&nbsp;Token&nbsp;Embedding</span></span></span></span></span></span>
<h2 class="anchor anchorTargetHideOnScrollNavbar_vjPI" id="end-to-end-transformer-workflow">End-to-end transformer workflow<a href="https://dheerajinampudi.com/blog/spaces/ai-space/attention-and-transformers#end-to-end-transformer-workflow" class="hash-link" aria-label="Direct link to End-to-end transformer workflow" title="Direct link to End-to-end transformer workflow" translate="no">​</a></h2>
<ol>
<li class=""><strong>Input processing:</strong>
<ul>
<li class="">Tokenize input (e.g., "The cat sat").</li>
<li class="">Add positional encodings.</li>
</ul>
</li>
<li class=""><strong>Encoder:</strong>
<ul>
<li class="">Processes input embeddings into contextual representations.</li>
</ul>
</li>
<li class=""><strong>Decoder:</strong>
<ul>
<li class="">Predicts the next token based on encoder outputs and already-generated tokens.</li>
</ul>
</li>
<li class=""><strong>Output:</strong>
<ul>
<li class="">Decodes token predictions into readable text.</li>
</ul>
</li>
</ol>
<h2 class="anchor anchorTargetHideOnScrollNavbar_vjPI" id="key-strengths-of-transformers">Key strengths of transformers<a href="https://dheerajinampudi.com/blog/spaces/ai-space/attention-and-transformers#key-strengths-of-transformers" class="hash-link" aria-label="Direct link to Key strengths of transformers" title="Direct link to Key strengths of transformers" translate="no">​</a></h2>
<ol>
<li class=""><strong>Handles long-range dependencies:</strong> Self-attention captures relationships between distant tokens.</li>
<li class=""><strong>Parallelizable:</strong> Processes sequences in parallel, unlike RNNs.</li>
<li class=""><strong>Scalable:</strong> Stacking multiple encoder/decoder blocks improves performance on large datasets.</li>
<li class=""><strong>Adaptable:</strong> Works across NLP, vision, and multimodal tasks.</li>
</ol>
<h2 class="anchor anchorTargetHideOnScrollNavbar_vjPI" id="next-steps">Next steps<a href="https://dheerajinampudi.com/blog/spaces/ai-space/attention-and-transformers#next-steps" class="hash-link" aria-label="Direct link to Next steps" title="Direct link to Next steps" translate="no">​</a></h2>
<ol>
<li class="">Experiment with <strong>attention matrices</strong> (visualizing weights).</li>
<li class="">Implement a small Transformer from scratch to solidify understanding.</li>
<li class="">Explore pre-trained models (e.g., BERT, GPT) to see Transformers in action.</li>
</ol>]]></content:encoded>
            <category>Spaces</category>
            <category>AI Space</category>
            <category>Transformers</category>
        </item>
        <item>
            <title><![CDATA[Quotes and phrases]]></title>
            <link>https://dheerajinampudi.com/blog/spaces/quotes/quotes</link>
            <guid>https://dheerajinampudi.com/blog/spaces/quotes/quotes</guid>
            <pubDate>Thu, 12 Sep 2024 00:00:00 GMT</pubDate>
            <description><![CDATA[Quotes, phrases and rules for life I have collected over the years.]]></description>
            <content:encoded><![CDATA[<hr>
<h2 class="anchor anchorTargetHideOnScrollNavbar_vjPI" id="naval-ravikant-rules">Naval Ravikant Rules<a href="https://dheerajinampudi.com/blog/spaces/quotes/quotes#naval-ravikant-rules" class="hash-link" aria-label="Direct link to Naval Ravikant Rules" title="Direct link to Naval Ravikant Rules" translate="no">​</a></h2>
<h3 class="anchor anchorTargetHideOnScrollNavbar_vjPI" id="rules-for-life-genius-rules"><strong>Rules for Life (<a href="https://www.youtube.com/watch?v=TmAO9jBqJf4" target="_blank" rel="noopener noreferrer" class="">Genius Rules</a>)</strong><a href="https://dheerajinampudi.com/blog/spaces/quotes/quotes#rules-for-life-genius-rules" class="hash-link" aria-label="Direct link to rules-for-life-genius-rules" title="Direct link to rules-for-life-genius-rules" translate="no">​</a></h3>
<ol>
<li class="">Be Ready to Start Over</li>
<li class="">When Inspired, Act Immediately</li>
<li class="">Choose to Be Yourself</li>
<li class="">Do What Feels Like Play to You, But Looks Like Work to Others</li>
<li class="">Choose Your Desires Carefully</li>
<li class="">Fall in Love with Reading</li>
<li class="">Master the Basics</li>
<li class="">Be an Owner</li>
<li class="">Use the New Leverage</li>
<li class="">Create Empty Space</li>
<li class="">Overcome the Need for External Validation</li>
</ol>
<blockquote>
<p>A busy calendar and a busy mind will destroy your ability to create anything great.</p>
</blockquote>
<hr>
<h2 class="anchor anchorTargetHideOnScrollNavbar_vjPI" id="general-quotes">General Quotes<a href="https://dheerajinampudi.com/blog/spaces/quotes/quotes#general-quotes" class="hash-link" aria-label="Direct link to General Quotes" title="Direct link to General Quotes" translate="no">​</a></h2>
<ul>
<li class="">"You must expect great things of yourself before you can do them." — Michael Jordan</li>
<li class="">"Any person capable of angering you becomes your master." — Epictetus</li>
<li class="">"The greatest sin is to think that you are weak." — Swami Vivekananda</li>
<li class="">"A question well put is half solved." — Charles Kettering</li>
<li class="">"Would you build a beautiful house and let someone you dislike live in it for free? Then why let them occupy space in your mind rent-free?" — Sadhguru</li>
</ul>
<hr>
<h2 class="anchor anchorTargetHideOnScrollNavbar_vjPI" id="personal-reflections">Personal Reflections<a href="https://dheerajinampudi.com/blog/spaces/quotes/quotes#personal-reflections" class="hash-link" aria-label="Direct link to Personal Reflections" title="Direct link to Personal Reflections" translate="no">​</a></h2>
<h3 class="anchor anchorTargetHideOnScrollNavbar_vjPI" id="own"><strong>Own</strong><a href="https://dheerajinampudi.com/blog/spaces/quotes/quotes#own" class="hash-link" aria-label="Direct link to own" title="Direct link to own" translate="no">​</a></h3>
<ul>
<li class="">"We must accept finite disappointment, but never lose infinite hope."</li>
<li class="">"When your own feelings override the facts, you will stay in the dark."</li>
<li class="">"A fool will get you to their level and win the argument with experience. Don’t fall for it."</li>
</ul>
<hr>
<h2 class="anchor anchorTargetHideOnScrollNavbar_vjPI" id="leadership-lessons">Leadership Lessons<a href="https://dheerajinampudi.com/blog/spaces/quotes/quotes#leadership-lessons" class="hash-link" aria-label="Direct link to Leadership Lessons" title="Direct link to Leadership Lessons" translate="no">​</a></h2>
<ul>
<li class="">"Having access to people smarter than you is a blessing, not a threat."</li>
<li class="">"When you focus on you, you grow. When you focus on shit, shit grows."</li>
<li class="">"Remember, you don't need to be aggressive. You need to be clear and concise."</li>
<li class="">"Jo dikhta hai, wahi bikta hai." <em>(What is visible, sells.)</em></li>
</ul>
<hr>
<h2 class="anchor anchorTargetHideOnScrollNavbar_vjPI" id="tech-principles">Tech Principles<a href="https://dheerajinampudi.com/blog/spaces/quotes/quotes#tech-principles" class="hash-link" aria-label="Direct link to Tech Principles" title="Direct link to Tech Principles" translate="no">​</a></h2>
<ul>
<li class=""><em>Quote from "Clean Architecture"</em>:<!-- -->
<blockquote>
<p>"Gather together the things that change for the same reasons. Separate those things that change for different reasons."</p>
</blockquote>
</li>
<li class="">"Don’t let the perfect get in the way of good enough."</li>
<li class="">"The smarter I get, the more I realize that I can’t be sure of much."</li>
<li class="">"As developers, we don't do things because they are easy; we do them because we thought they were going to be easy."</li>
</ul>
<hr>
<h3 class="anchor anchorTargetHideOnScrollNavbar_vjPI" id="business"><strong>Business</strong><a href="https://dheerajinampudi.com/blog/spaces/quotes/quotes#business" class="hash-link" aria-label="Direct link to business" title="Direct link to business" translate="no">​</a></h3>
<ul>
<li class="">"Businesses today are data rich and insights poor."</li>
</ul>
<hr>
<h2 class="anchor anchorTargetHideOnScrollNavbar_vjPI" id="starting-over-3-things-to-remember">Starting Over: 3 Things to Remember<a href="https://dheerajinampudi.com/blog/spaces/quotes/quotes#starting-over-3-things-to-remember" class="hash-link" aria-label="Direct link to Starting Over: 3 Things to Remember" title="Direct link to Starting Over: 3 Things to Remember" translate="no">​</a></h2>
<ol>
<li class=""><strong>People who did it before you aren’t super smarter than you.</strong></li>
<li class=""><strong>You’re not bad at it; you’re just starting out.</strong></li>
<li class=""><strong>There’s nothing you need to know that you can’t find out or figure out.</strong></li>
</ol>]]></content:encoded>
            <category>Spaces</category>
            <category>Quotes</category>
        </item>
        <item>
            <title><![CDATA[Research paper picks]]></title>
            <link>https://dheerajinampudi.com/blog/spaces/ai-space/research-paper-picks</link>
            <guid>https://dheerajinampudi.com/blog/spaces/ai-space/research-paper-picks</guid>
            <pubDate>Thu, 22 Aug 2024 00:00:00 GMT</pubDate>
            <description><![CDATA[Eleven August 2024 arXiv papers on GraphRAG, coding agents, spiking networks, multi-agent RL and new views of attention.]]></description>
            <content:encoded><![CDATA[<p>A reading list of August 2024 arXiv papers, grouped by theme. Each entry links to the paper PDF with an excerpt from its abstract; my own comments are marked as notes. Source: code_your_own_AI.</p>
<h2 class="anchor anchorTargetHideOnScrollNavbar_vjPI" id="retrieval-and-knowledge-graphs">Retrieval and knowledge graphs<a href="https://dheerajinampudi.com/blog/spaces/ai-space/research-paper-picks#retrieval-and-knowledge-graphs" class="hash-link" aria-label="Direct link to Retrieval and knowledge graphs" title="Direct link to Retrieval and knowledge graphs" translate="no">​</a></h2>
<h3 class="anchor anchorTargetHideOnScrollNavbar_vjPI" id="graph-retrieval-augmented-generation-a-survey">Graph Retrieval-Augmented Generation: A Survey<a href="https://dheerajinampudi.com/blog/spaces/ai-space/research-paper-picks#graph-retrieval-augmented-generation-a-survey" class="hash-link" aria-label="Direct link to Graph Retrieval-Augmented Generation: A Survey" title="Direct link to Graph Retrieval-Augmented Generation: A Survey" translate="no">​</a></h3>
<p><a href="https://arxiv.org/pdf/2408.08921" target="_blank" rel="noopener noreferrer" class="">arxiv.org/pdf/2408.08921</a></p>
<blockquote>
<p>GraphRAG leverages structural information across entities to enable more precise and comprehensive retrieval, capturing relational knowledge and facilitating more accurate, context-aware responses. Given the novelty and potential of GraphRAG, a systematic review of current technologies is imperative. This paper provides the first comprehensive overview of GraphRAG methodologies. We formalize the GraphRAG workflow, encompassing Graph-Based Indexing, Graph-Guided Retrieval, and Graph-Enhanced Generation. We then outline the core technologies and training methods at each stage.</p>
</blockquote>
<h2 class="anchor anchorTargetHideOnScrollNavbar_vjPI" id="agents-reasoning-and-prompting">Agents, reasoning and prompting<a href="https://dheerajinampudi.com/blog/spaces/ai-space/research-paper-picks#agents-reasoning-and-prompting" class="hash-link" aria-label="Direct link to Agents, reasoning and prompting" title="Direct link to Agents, reasoning and prompting" translate="no">​</a></h2>
<h3 class="anchor anchorTargetHideOnScrollNavbar_vjPI" id="verilogcoder-autonomous-verilog-coding-agents-with-graph-based-planning-and-abstract-syntax-tree-ast-based-waveform-tracing-tool">VerilogCoder: Autonomous Verilog Coding Agents with Graph-based Planning and Abstract Syntax Tree (AST)-based Waveform Tracing Tool<a href="https://dheerajinampudi.com/blog/spaces/ai-space/research-paper-picks#verilogcoder-autonomous-verilog-coding-agents-with-graph-based-planning-and-abstract-syntax-tree-ast-based-waveform-tracing-tool" class="hash-link" aria-label="Direct link to VerilogCoder: Autonomous Verilog Coding Agents with Graph-based Planning and Abstract Syntax Tree (AST)-based Waveform Tracing Tool" title="Direct link to VerilogCoder: Autonomous Verilog Coding Agents with Graph-based Planning and Abstract Syntax Tree (AST)-based Waveform Tracing Tool" translate="no">​</a></h3>
<p><a href="https://arxiv.org/pdf/2408.08927" target="_blank" rel="noopener noreferrer" class="">arxiv.org/pdf/2408.08927</a></p>
<blockquote>
<p>Verilog is a popular hardware description language for designing and modeling digital systems; thus, Verilog generation is one of the emerging areas of research to facilitate the design process. In this work, we propose VerilogCoder, a system of multiple Artificial Intelligence (AI) agents for Verilog code generation, to autonomously write Verilog code and fix syntax and functional errors using collaborative Verilog tools (i.e., syntax checker, simulator, and waveform tracer).</p>
</blockquote>
<h3 class="anchor anchorTargetHideOnScrollNavbar_vjPI" id="cognitive-llms-towards-integrating-cognitive-architectures-and-large-language-models-for-manufacturing-decision-making">Cognitive LLMs: Towards Integrating Cognitive Architectures and Large Language Models for Manufacturing Decision-making<a href="https://dheerajinampudi.com/blog/spaces/ai-space/research-paper-picks#cognitive-llms-towards-integrating-cognitive-architectures-and-large-language-models-for-manufacturing-decision-making" class="hash-link" aria-label="Direct link to Cognitive LLMs: Towards Integrating Cognitive Architectures and Large Language Models for Manufacturing Decision-making" title="Direct link to Cognitive LLMs: Towards Integrating Cognitive Architectures and Large Language Models for Manufacturing Decision-making" translate="no">​</a></h3>
<p><a href="https://arxiv.org/pdf/2408.09176" target="_blank" rel="noopener noreferrer" class="">arxiv.org/pdf/2408.09176</a></p>
<blockquote>
<p>Advancements in multi-agent LLM frameworks as well as emergent capabilities such as in-context learning have enabled LLMs to employ more sophisticated reasoning strategies, such as 'chain-of-thought' reasoning (CoT). We propose LLM-ACTR, which shows improved decision-making capabilities over LLMs by integrating intermediate representations extracted from a well-establish neuro-symbolic system: the ACT-R cognitive architecture.</p>
</blockquote>
<h3 class="anchor anchorTargetHideOnScrollNavbar_vjPI" id="neuro-symbolic-ai-for-military-applications">Neuro-Symbolic AI for Military Applications<a href="https://dheerajinampudi.com/blog/spaces/ai-space/research-paper-picks#neuro-symbolic-ai-for-military-applications" class="hash-link" aria-label="Direct link to Neuro-Symbolic AI for Military Applications" title="Direct link to Neuro-Symbolic AI for Military Applications" translate="no">​</a></h3>
<p><a href="https://arxiv.org/pdf/2408.09224" target="_blank" rel="noopener noreferrer" class="">arxiv.org/pdf/2408.09224</a></p>
<div class="theme-admonition theme-admonition-note admonition_xJq3 alert alert--secondary"><div class="admonitionHeading_Gvgb"><span class="admonitionIcon_Rf37"><svg viewBox="0 0 14 16"><path fill-rule="evenodd" d="M6.3 5.69a.942.942 0 0 1-.28-.7c0-.28.09-.52.28-.7.19-.18.42-.28.7-.28.28 0 .52.09.7.28.18.19.28.42.28.7 0 .28-.09.52-.28.7a1 1 0 0 1-.7.3c-.28 0-.52-.11-.7-.3zM8 7.99c-.02-.25-.11-.48-.31-.69-.2-.19-.42-.3-.69-.31H6c-.27.02-.48.13-.69.31-.2.2-.3.44-.31.69h1v3c.02.27.11.5.31.69.2.2.42.31.69.31h1c.27 0 .48-.11.69-.31.2-.19.3-.42.31-.69H8V7.98v.01zM7 2.3c-3.14 0-5.7 2.54-5.7 5.68 0 3.14 2.56 5.7 5.7 5.7s5.7-2.55 5.7-5.7c0-3.15-2.56-5.69-5.7-5.69v.01zM7 .98c3.86 0 7 3.14 7 7s-3.14 7-7 7-7-3.12-7-7 3.14-7 7-7z"></path></svg></span>note</div><div class="admonitionContent_BuS1"><p>This is an exception: I'm including a paper on military defence because it explains the civilian possibilities offered by neuro-symbolic AI. Otherwise, I try to keep military context out of my content.</p></div></div>
<blockquote>
<p>Contributing to the growing body of research, this study represents a comprehensive exploration of the extensive possibilities offered by Neuro-Symbolic AI.</p>
</blockquote>
<h3 class="anchor anchorTargetHideOnScrollNavbar_vjPI" id="concept-distillation-from-strong-to-weak-models-via-hypotheses-to-theories-prompting">Concept Distillation from Strong to Weak Models via Hypotheses-to-Theories Prompting<a href="https://dheerajinampudi.com/blog/spaces/ai-space/research-paper-picks#concept-distillation-from-strong-to-weak-models-via-hypotheses-to-theories-prompting" class="hash-link" aria-label="Direct link to Concept Distillation from Strong to Weak Models via Hypotheses-to-Theories Prompting" title="Direct link to Concept Distillation from Strong to Weak Models via Hypotheses-to-Theories Prompting" translate="no">​</a></h3>
<p><a href="https://arxiv.org/pdf/2408.09365" target="_blank" rel="noopener noreferrer" class="">arxiv.org/pdf/2408.09365</a></p>
<div class="theme-admonition theme-admonition-note admonition_xJq3 alert alert--secondary"><div class="admonitionHeading_Gvgb"><span class="admonitionIcon_Rf37"><svg viewBox="0 0 14 16"><path fill-rule="evenodd" d="M6.3 5.69a.942.942 0 0 1-.28-.7c0-.28.09-.52.28-.7.19-.18.42-.28.7-.28.28 0 .52.09.7.28.18.19.28.42.28.7 0 .28-.09.52-.28.7a1 1 0 0 1-.7.3c-.28 0-.52-.11-.7-.3zM8 7.99c-.02-.25-.11-.48-.31-.69-.2-.19-.42-.3-.69-.31H6c-.27.02-.48.13-.69.31-.2.2-.3.44-.31.69h1v3c.02.27.11.5.31.69.2.2.42.31.69.31h1c.27 0 .48-.11.69-.31.2-.19.3-.42.31-.69H8V7.98v.01zM7 2.3c-3.14 0-5.7 2.54-5.7 5.68 0 3.14 2.56 5.7 5.7 5.7s5.7-2.55 5.7-5.7c0-3.15-2.56-5.69-5.7-5.69v.01zM7 .98c3.86 0 7 3.14 7 7s-3.14 7-7 7-7-3.12-7-7 3.14-7 7-7z"></path></svg></span>note</div><div class="admonitionContent_BuS1"><p>Microsoft hides its automatic prompt optimization technique for enhancing weaker models on complex tasks here.</p></div></div>
<blockquote>
<p>Hand-crafting high quality prompts to optimize the performance of language models is a complicated and labor-intensive process. We propose Concept Distillation (CD), an automatic prompt optimization technique for enhancing weaker models on complex tasks.</p>
</blockquote>
<h3 class="anchor anchorTargetHideOnScrollNavbar_vjPI" id="multi-agent-reinforcement-learning-for-autonomous-driving-a-survey">Multi-Agent Reinforcement Learning for Autonomous Driving: A Survey<a href="https://dheerajinampudi.com/blog/spaces/ai-space/research-paper-picks#multi-agent-reinforcement-learning-for-autonomous-driving-a-survey" class="hash-link" aria-label="Direct link to Multi-Agent Reinforcement Learning for Autonomous Driving: A Survey" title="Direct link to Multi-Agent Reinforcement Learning for Autonomous Driving: A Survey" translate="no">​</a></h3>
<p><a href="https://arxiv.org/pdf/2408.09675" target="_blank" rel="noopener noreferrer" class="">arxiv.org/pdf/2408.09675</a></p>
<blockquote>
<p>Reinforcement Learning (RL) is a potent tool for sequential decision-making and has achieved performance surpassing human capabilities across many challenging real-world tasks. As the extension of RL in the multi-agent system domain, multi-agent RL (MARL) not only need to learn the control policy but also requires consideration regarding interactions with all other agents in the environment, mutual influences among different system components, and the distribution of computational resources.</p>
</blockquote>
<h2 class="anchor anchorTargetHideOnScrollNavbar_vjPI" id="spiking-neural-networks">Spiking neural networks<a href="https://dheerajinampudi.com/blog/spaces/ai-space/research-paper-picks#spiking-neural-networks" class="hash-link" aria-label="Direct link to Spiking neural networks" title="Direct link to Spiking neural networks" translate="no">​</a></h2>
<h3 class="anchor anchorTargetHideOnScrollNavbar_vjPI" id="temporal-reversed-training-for-spiking-neural-networks-with-generalized-spatio-temporal-representation">Temporal Reversed Training for Spiking Neural Networks with Generalized Spatio-Temporal Representation<a href="https://dheerajinampudi.com/blog/spaces/ai-space/research-paper-picks#temporal-reversed-training-for-spiking-neural-networks-with-generalized-spatio-temporal-representation" class="hash-link" aria-label="Direct link to Temporal Reversed Training for Spiking Neural Networks with Generalized Spatio-Temporal Representation" title="Direct link to Temporal Reversed Training for Spiking Neural Networks with Generalized Spatio-Temporal Representation" translate="no">​</a></h3>
<p><a href="https://arxiv.org/pdf/2408.09108" target="_blank" rel="noopener noreferrer" class="">arxiv.org/pdf/2408.09108</a></p>
<blockquote>
<p>Brain-inspired spiking neural networks (SNNs) have received widespread attention. Unlike traditional artificial neural networks (ANNs), which transfer information using intensive floating-point values, SNNs transfer discrete 0-1 spikes between neurons, providing a more efficient neuromorphic computing paradigm.</p>
</blockquote>
<h3 class="anchor anchorTargetHideOnScrollNavbar_vjPI" id="obtaining-optimal-spiking-neural-network-in-sequence-learning-via-crnn-snn-conversion">Obtaining Optimal Spiking Neural Network in Sequence Learning via CRNN-SNN Conversion<a href="https://dheerajinampudi.com/blog/spaces/ai-space/research-paper-picks#obtaining-optimal-spiking-neural-network-in-sequence-learning-via-crnn-snn-conversion" class="hash-link" aria-label="Direct link to Obtaining Optimal Spiking Neural Network in Sequence Learning via CRNN-SNN Conversion" title="Direct link to Obtaining Optimal Spiking Neural Network in Sequence Learning via CRNN-SNN Conversion" translate="no">​</a></h3>
<p><a href="https://arxiv.org/pdf/2408.09403" target="_blank" rel="noopener noreferrer" class="">arxiv.org/pdf/2408.09403</a></p>
<blockquote>
<p>Spiking neural networks (SNNs) are becoming a promising alternative to conventional artificial neural networks (ANNs) due to their rich neural dynamics and the implementation of energy-efficient neuromorphic chips.</p>
</blockquote>
<h2 class="anchor anchorTargetHideOnScrollNavbar_vjPI" id="attention-and-sequence-modeling">Attention and sequence modeling<a href="https://dheerajinampudi.com/blog/spaces/ai-space/research-paper-picks#attention-and-sequence-modeling" class="hash-link" aria-label="Direct link to Attention and sequence modeling" title="Direct link to Attention and sequence modeling" translate="no">​</a></h2>
<h3 class="anchor anchorTargetHideOnScrollNavbar_vjPI" id="elastic-efficient-linear-attention-for-sequential-interest-compression">ELASTIC: Efficient Linear Attention for Sequential Interest Compression<a href="https://dheerajinampudi.com/blog/spaces/ai-space/research-paper-picks#elastic-efficient-linear-attention-for-sequential-interest-compression" class="hash-link" aria-label="Direct link to ELASTIC: Efficient Linear Attention for Sequential Interest Compression" title="Direct link to ELASTIC: Efficient Linear Attention for Sequential Interest Compression" translate="no">​</a></h3>
<p><a href="https://arxiv.org/pdf/2408.09380" target="_blank" rel="noopener noreferrer" class="">arxiv.org/pdf/2408.09380</a></p>
<div class="theme-admonition theme-admonition-note admonition_xJq3 alert alert--secondary"><div class="admonitionHeading_Gvgb"><span class="admonitionIcon_Rf37"><svg viewBox="0 0 14 16"><path fill-rule="evenodd" d="M6.3 5.69a.942.942 0 0 1-.28-.7c0-.28.09-.52.28-.7.19-.18.42-.28.7-.28.28 0 .52.09.7.28.18.19.28.42.28.7 0 .28-.09.52-.28.7a1 1 0 0 1-.7.3c-.28 0-.52-.11-.7-.3zM8 7.99c-.02-.25-.11-.48-.31-.69-.2-.19-.42-.3-.69-.31H6c-.27.02-.48.13-.69.31-.2.2-.3.44-.31.69h1v3c.02.27.11.5.31.69.2.2.42.31.69.31h1c.27 0 .48-.11.69-.31.2-.19.3-.42.31-.69H8V7.98v.01zM7 2.3c-3.14 0-5.7 2.54-5.7 5.68 0 3.14 2.56 5.7 5.7 5.7s5.7-2.55 5.7-5.7c0-3.15-2.56-5.69-5.7-5.69v.01zM7 .98c3.86 0 7 3.14 7 7s-3.14 7-7 7-7-3.12-7-7 3.14-7 7-7z"></path></svg></span>note</div><div class="admonitionContent_BuS1"><p>A new update on the transformer's attention mechanism.</p></div></div>
<blockquote>
<p>The quadratic computational and memory complexities of self attention have limited its scalability for modeling users' long range behaviour sequences. To address this problem, we propose ELASTIC, an Efficient Linear Attention for SequenTial Interest Compression, requiring only linear time complexity and decoupling model capacity from computational cost.</p>
</blockquote>
<h3 class="anchor anchorTargetHideOnScrollNavbar_vjPI" id="attention-is-a-smoothed-cubic-spline">Attention is a smoothed cubic spline<a href="https://dheerajinampudi.com/blog/spaces/ai-space/research-paper-picks#attention-is-a-smoothed-cubic-spline" class="hash-link" aria-label="Direct link to Attention is a smoothed cubic spline" title="Direct link to Attention is a smoothed cubic spline" translate="no">​</a></h3>
<p><a href="https://arxiv.org/pdf/2408.09624" target="_blank" rel="noopener noreferrer" class="">arxiv.org/pdf/2408.09624</a></p>
<blockquote>
<p>The attention module in a transformer is a smoothed cubic spline. Viewed in this manner, this mysterious but critical component of a transformer becomes a natural development of an old notion deeply entrenched in classical approximation theory.</p>
</blockquote>
<h3 class="anchor anchorTargetHideOnScrollNavbar_vjPI" id="siamese-multiple-attention-temporal-convolution-networks-for-human-mobility-signature-identification">Siamese Multiple Attention Temporal Convolution Networks for Human Mobility Signature Identification<a href="https://dheerajinampudi.com/blog/spaces/ai-space/research-paper-picks#siamese-multiple-attention-temporal-convolution-networks-for-human-mobility-signature-identification" class="hash-link" aria-label="Direct link to Siamese Multiple Attention Temporal Convolution Networks for Human Mobility Signature Identification" title="Direct link to Siamese Multiple Attention Temporal Convolution Networks for Human Mobility Signature Identification" translate="no">​</a></h3>
<p><a href="https://arxiv.org/pdf/2408.09230" target="_blank" rel="noopener noreferrer" class="">arxiv.org/pdf/2408.09230</a></p>
<blockquote>
<p>We propose a Siamese Multiple Attention Temporal Convolutional Network (Siamese MA-TCN) to capitalize on the strengths of both TCN architecture and multi-head self-attention, enabling the proficient extraction of both local and long-term dependencies.</p>
</blockquote>]]></content:encoded>
            <category>Spaces</category>
            <category>AI Space</category>
            <category>Research Papers</category>
        </item>
        <item>
            <title><![CDATA[Mental models]]></title>
            <link>https://dheerajinampudi.com/blog/spaces/mental-models/mental-models</link>
            <guid>https://dheerajinampudi.com/blog/spaces/mental-models/mental-models</guid>
            <pubDate>Mon, 05 Aug 2024 00:00:00 GMT</pubDate>
            <description><![CDATA[My own mental models, plus the classic ones I use for everyday decisions.]]></description>
            <content:encoded><![CDATA[<p>Mental models are powerful tools for understanding the world, making better decisions, and solving problems. Think of them as mental shortcuts or frameworks that help you navigate complexity.</p>
<h2 class="anchor anchorTargetHideOnScrollNavbar_vjPI" id="my-own-mental-models">My own mental models<a href="https://dheerajinampudi.com/blog/spaces/mental-models/mental-models#my-own-mental-models" class="hash-link" aria-label="Direct link to My own mental models" title="Direct link to My own mental models" translate="no">​</a></h2>
<p>I write these up as short posts on the Blog, one idea at a time:</p>
<ul>
<li class=""><a class="" href="https://dheerajinampudi.com/blog/mental-models/dont-hand-over-the-controls">Don't hand over the controls</a>: when someone's words make you angry or ready to give up.</li>
<li class=""><a class="" href="https://dheerajinampudi.com/blog/mental-models/no-free-rent-in-your-mind">No free rent in your mind</a>: when someone you dislike keeps occupying your thoughts.</li>
</ul>
<p>See the full series under <a class="" href="https://dheerajinampudi.com/blog/tags/mental-models">Mental Models on the Blog</a>.</p>
<h2 class="anchor anchorTargetHideOnScrollNavbar_vjPI" id="must-know-mental-models-for-everyday-life">Must-Know Mental Models for Everyday Life<a href="https://dheerajinampudi.com/blog/spaces/mental-models/mental-models#must-know-mental-models-for-everyday-life" class="hash-link" aria-label="Direct link to Must-Know Mental Models for Everyday Life" title="Direct link to Must-Know Mental Models for Everyday Life" translate="no">​</a></h2>
<h3 class="anchor anchorTargetHideOnScrollNavbar_vjPI" id="1-how-to-make-smart-decisions-without-getting-lucky">1. How to Make Smart Decisions Without Getting Lucky<a href="https://dheerajinampudi.com/blog/spaces/mental-models/mental-models#1-how-to-make-smart-decisions-without-getting-lucky" class="hash-link" aria-label="Direct link to 1. How to Make Smart Decisions Without Getting Lucky" title="Direct link to 1. How to Make Smart Decisions Without Getting Lucky" translate="no">​</a></h3>
<ul>
<li class=""><strong>Key Idea:</strong> Smart decisions aren't based on luck. They come from understanding probabilities, risks, and potential outcomes.</li>
<li class=""><strong>Learn More:</strong> <a href="https://fs.blog/smart-decisions/" target="_blank" rel="noopener noreferrer" class="">How to Make Smart Decisions Without Getting Lucky</a></li>
</ul>
<h3 class="anchor anchorTargetHideOnScrollNavbar_vjPI" id="2-the-feynman-learning-technique">2. The Feynman Learning Technique<a href="https://dheerajinampudi.com/blog/spaces/mental-models/mental-models#2-the-feynman-learning-technique" class="hash-link" aria-label="Direct link to 2. The Feynman Learning Technique" title="Direct link to 2. The Feynman Learning Technique" translate="no">​</a></h3>
<ul>
<li class=""><strong>Key Idea:</strong> Learning is not just about memorization. To truly understand something, explain it simply, identify gaps in your knowledge, and refine your understanding.</li>
<li class=""><strong>Learn More:</strong> <a href="https://jamesclear.com/" target="_blank" rel="noopener noreferrer" class="">The Feynman Learning Technique</a></li>
</ul>
<h3 class="anchor anchorTargetHideOnScrollNavbar_vjPI" id="3-understanding-your-circle-of-competence">3. Understanding Your Circle of Competence<a href="https://dheerajinampudi.com/blog/spaces/mental-models/mental-models#3-understanding-your-circle-of-competence" class="hash-link" aria-label="Direct link to 3. Understanding Your Circle of Competence" title="Direct link to 3. Understanding Your Circle of Competence" translate="no">​</a></h3>
<ul>
<li class=""><strong>Key Idea:</strong> Know your strengths and limitations. Focus on areas where you have expertise and avoid overconfidence in areas where you don't.</li>
<li class=""><strong>Learn More:</strong> <a href="https://fs.blog/circle-of-competence/" target="_blank" rel="noopener noreferrer" class="">Understanding your Circle of Competence: Avoiding Problems</a></li>
</ul>
<h2 class="anchor anchorTargetHideOnScrollNavbar_vjPI" id="mental-models-for-making-decisions">Mental Models for Making Decisions<a href="https://dheerajinampudi.com/blog/spaces/mental-models/mental-models#mental-models-for-making-decisions" class="hash-link" aria-label="Direct link to Mental Models for Making Decisions" title="Direct link to Mental Models for Making Decisions" translate="no">​</a></h2>
<h3 class="anchor anchorTargetHideOnScrollNavbar_vjPI" id="4-mental-models-the-best-way-to-make-intelligent-decisions-100-models-explained">4. Mental Models: The Best Way to Make Intelligent Decisions (~100 Models Explained)<a href="https://dheerajinampudi.com/blog/spaces/mental-models/mental-models#4-mental-models-the-best-way-to-make-intelligent-decisions-100-models-explained" class="hash-link" aria-label="Direct link to 4. Mental Models: The Best Way to Make Intelligent Decisions (~100 Models Explained)" title="Direct link to 4. Mental Models: The Best Way to Make Intelligent Decisions (~100 Models Explained)" translate="no">​</a></h3>
<ul>
<li class=""><strong>Key Idea:</strong> This comprehensive guide explores a wide range of mental models from various fields. Learn how to apply these models to improve decision-making.</li>
<li class=""><strong>Learn More:</strong> <a href="https://en.wikipedia.org/wiki/Mental_model" target="_blank" rel="noopener noreferrer" class="">Mental Models: The Best Way to Make Intelligent Decisions (~100 Models Explained)</a></li>
</ul>
<h2 class="anchor anchorTargetHideOnScrollNavbar_vjPI" id="mental-models-for-startup-founders">Mental Models for Startup Founders<a href="https://dheerajinampudi.com/blog/spaces/mental-models/mental-models#mental-models-for-startup-founders" class="hash-link" aria-label="Direct link to Mental Models for Startup Founders" title="Direct link to Mental Models for Startup Founders" translate="no">​</a></h2>
<h3 class="anchor anchorTargetHideOnScrollNavbar_vjPI" id="5-mental-models-for-startup-founders-a-free-book">5. Mental models for startup founders: a free book<a href="https://dheerajinampudi.com/blog/spaces/mental-models/mental-models#5-mental-models-for-startup-founders-a-free-book" class="hash-link" aria-label="Direct link to 5. Mental models for startup founders: a free book" title="Direct link to 5. Mental models for startup founders: a free book" translate="no">​</a></h3>
<ul>
<li class=""><strong>Key Idea:</strong> This free book provides a curated collection of mental models specifically tailored for startup founders. Discover how to apply these models to navigate the challenges of entrepreneurship.</li>
<li class=""><strong>Learn More:</strong> <a href="https://guustaaf.com/naval/Naval-HowToGetRich.pdf" target="_blank" rel="noopener noreferrer" class="">Mental models for startup founders: a free book</a></li>
</ul>
<h2 class="anchor anchorTargetHideOnScrollNavbar_vjPI" id="more-resources">More Resources<a href="https://dheerajinampudi.com/blog/spaces/mental-models/mental-models#more-resources" class="hash-link" aria-label="Direct link to More Resources" title="Direct link to More Resources" translate="no">​</a></h2>
<ul>
<li class=""><strong>Nav.al:</strong> A treasure trove of insights from Naval Ravikant, a successful entrepreneur and investor. Explore his ideas on wealth, happiness, and decision-making.</li>
<li class=""><strong>Farnam Street Blog:</strong> This blog delves into mental models, decision-making, and critical thinking skills.</li>
</ul>
<p><strong>Remember:</strong> The best way to use mental models is to practice applying them to real-life situations. Start with a few that resonate with you and gradually expand your toolkit.</p>]]></content:encoded>
            <category>Spaces</category>
            <category>Mental Models</category>
        </item>
        <item>
            <title><![CDATA[RAG approach in trading]]></title>
            <link>https://dheerajinampudi.com/blog/spaces/ai-space/rag-in-trading</link>
            <guid>https://dheerajinampudi.com/blog/spaces/ai-space/rag-in-trading</guid>
            <pubDate>Sat, 03 Aug 2024 00:00:00 GMT</pubDate>
            <description><![CDATA[A reference design for using RAG in trading, from live news and market feeds to generated insights and brokerage order execution.]]></description>
            <content:encoded><![CDATA[<p>Using retrieval-augmented generation (RAG) in trading means integrating live data, such as news and economic indicators, with AI agents and advanced AI models to make informed trading decisions.</p>
<p>This note covers the components you need, the end-to-end workflow, an example implementation in Python, and the considerations to keep in mind.</p>
<h2 class="anchor anchorTargetHideOnScrollNavbar_vjPI" id="components-required">Components required<a href="https://dheerajinampudi.com/blog/spaces/ai-space/rag-in-trading#components-required" class="hash-link" aria-label="Direct link to Components required" title="Direct link to Components required" translate="no">​</a></h2>
<ol>
<li class="">
<p><strong>Live Data Feeds:</strong></p>
<ul>
<li class=""><strong>News APIs:</strong> Use APIs like NewsAPI, Alpha Vantage News, or any specific financial news provider to get real-time news updates.</li>
<li class=""><strong>Stock Market APIs:</strong> Use APIs like Alpha Vantage, IEX Cloud, or Yahoo Finance to get live stock prices and other financial metrics.</li>
</ul>
</li>
<li class="">
<p><strong>Trading Platform APIs:</strong></p>
<ul>
<li class=""><strong>Brokerage APIs:</strong> Integrate with brokerage APIs like Alpaca, Interactive Brokers, or Robinhood to execute buy/sell orders programmatically.</li>
</ul>
</li>
<li class="">
<p><strong>RAG Model:</strong></p>
<ul>
<li class=""><strong>Retrieval Component:</strong> Use Elasticsearch or another retrieval system to index and retrieve relevant news articles and financial data based on queries.</li>
<li class=""><strong>Generation Component:</strong> Use a transformer-based model like GPT-3 or a custom fine-tuned model to analyze the retrieved information and generate insights or trading signals.</li>
</ul>
</li>
<li class="">
<p><strong>Trading Algorithms:</strong></p>
<ul>
<li class="">Develop or use existing trading algorithms that can process the insights generated by the RAG model to make buy/sell decisions.</li>
</ul>
</li>
</ol>
<h2 class="anchor anchorTargetHideOnScrollNavbar_vjPI" id="workflow">Workflow<a href="https://dheerajinampudi.com/blog/spaces/ai-space/rag-in-trading#workflow" class="hash-link" aria-label="Direct link to Workflow" title="Direct link to Workflow" translate="no">​</a></h2>
<ol>
<li class="">
<p><strong>Data Retrieval:</strong></p>
<ul>
<li class="">Set up a pipeline to continuously fetch live news articles and stock prices using the respective APIs.</li>
<li class="">Index the news articles and other relevant data using a retrieval system like Elasticsearch.</li>
</ul>
</li>
<li class="">
<p><strong>Query Generation:</strong></p>
<ul>
<li class="">Create a query generator that formulates relevant queries based on the current stock market context. For example, the query could be "impact of current events on tech stocks."</li>
</ul>
</li>
<li class="">
<p><strong>Retrieval Process:</strong></p>
<ul>
<li class="">Use the retrieval component to fetch relevant news articles and data based on the generated queries.</li>
</ul>
</li>
<li class="">
<p><strong>Generation of Insights:</strong></p>
<ul>
<li class="">Use the generation component of the RAG model to analyze the retrieved information and generate trading insights. For example, the model could summarize how recent news might affect certain stocks.</li>
</ul>
</li>
<li class="">
<p><strong>Trading Algorithm Integration:</strong></p>
<ul>
<li class="">Integrate the insights with your trading algorithms. The trading algorithm should be designed to interpret the insights and make buy/sell decisions.</li>
<li class="">For example, if the RAG model indicates positive sentiment around a stock, the trading algorithm might decide to buy that stock.</li>
</ul>
</li>
<li class="">
<p><strong>Execution of Trades:</strong></p>
<ul>
<li class="">Use the brokerage API to execute the trades based on the decisions made by the trading algorithm.</li>
</ul>
</li>
</ol>
<h2 class="anchor anchorTargetHideOnScrollNavbar_vjPI" id="example-implementation">Example implementation<a href="https://dheerajinampudi.com/blog/spaces/ai-space/rag-in-trading#example-implementation" class="hash-link" aria-label="Direct link to Example implementation" title="Direct link to Example implementation" translate="no">​</a></h2>
<p>A minimal Python sketch of the loop: fetch and index news, generate insights, then act on them.</p>
<div class="language-python codeBlockContainer_Ckt0 theme-code-block" style="--prism-color:#393A34;--prism-background-color:#f6f8fa"><div class="codeBlockContent_QJqH"><pre tabindex="0" class="prism-code language-python codeBlock_bY9V thin-scrollbar" style="color:#393A34;background-color:#f6f8fa"><code class="codeBlockLines_e6Vv"><div class="token-line" style="color:#393A34"><span class="token keyword" style="color:#00009f">import</span><span class="token plain"> requests</span><br></div><div class="token-line" style="color:#393A34"><span class="token plain"></span><span class="token keyword" style="color:#00009f">from</span><span class="token plain"> elasticsearch </span><span class="token keyword" style="color:#00009f">import</span><span class="token plain"> Elasticsearch</span><br></div><div class="token-line" style="color:#393A34"><span class="token plain"></span><span class="token keyword" style="color:#00009f">import</span><span class="token plain"> openai</span><br></div><div class="token-line" style="color:#393A34"><span class="token plain" style="display:inline-block"></span><br></div><div class="token-line" style="color:#393A34"><span class="token plain"></span><span class="token comment" style="color:#999988;font-style:italic"># Configure APIs and Elasticsearch</span><span class="token plain"></span><br></div><div class="token-line" style="color:#393A34"><span class="token plain">news_api_url </span><span class="token operator" style="color:#393A34">=</span><span class="token plain"> </span><span class="token string" style="color:#e3116c">"https://newsapi.org/v2/everything"</span><span class="token plain"></span><br></div><div class="token-line" style="color:#393A34"><span class="token plain">stock_api_url </span><span class="token operator" style="color:#393A34">=</span><span class="token plain"> </span><span class="token string" style="color:#e3116c">"https://www.alphavantage.co/query"</span><span class="token plain"></span><br></div><div class="token-line" style="color:#393A34"><span class="token plain">broker_api_url </span><span class="token operator" style="color:#393A34">=</span><span class="token plain"> </span><span class="token string" style="color:#e3116c">"https://broker-api.example.com/trade"</span><span class="token plain"></span><br></div><div class="token-line" style="color:#393A34"><span class="token plain">es </span><span class="token operator" style="color:#393A34">=</span><span class="token plain"> Elasticsearch</span><span class="token punctuation" style="color:#393A34">(</span><span class="token string" style="color:#e3116c">"http://localhost:9200"</span><span class="token punctuation" style="color:#393A34">)</span><span class="token plain"></span><br></div><div class="token-line" style="color:#393A34"><span class="token plain" style="display:inline-block"></span><br></div><div class="token-line" style="color:#393A34"><span class="token plain"></span><span class="token comment" style="color:#999988;font-style:italic"># Fetch live news and stock prices</span><span class="token plain"></span><br></div><div class="token-line" style="color:#393A34"><span class="token plain"></span><span class="token keyword" style="color:#00009f">def</span><span class="token plain"> </span><span class="token function" style="color:#d73a49">fetch_news</span><span class="token punctuation" style="color:#393A34">(</span><span class="token plain">api_key</span><span class="token punctuation" style="color:#393A34">,</span><span class="token plain"> query</span><span class="token punctuation" style="color:#393A34">)</span><span class="token punctuation" style="color:#393A34">:</span><span class="token plain"></span><br></div><div class="token-line" style="color:#393A34"><span class="token plain">    response </span><span class="token operator" style="color:#393A34">=</span><span class="token plain"> requests</span><span class="token punctuation" style="color:#393A34">.</span><span class="token plain">get</span><span class="token punctuation" style="color:#393A34">(</span><span class="token plain">news_api_url</span><span class="token punctuation" style="color:#393A34">,</span><span class="token plain"> params</span><span class="token operator" style="color:#393A34">=</span><span class="token punctuation" style="color:#393A34">{</span><span class="token string" style="color:#e3116c">"q"</span><span class="token punctuation" style="color:#393A34">:</span><span class="token plain"> query</span><span class="token punctuation" style="color:#393A34">,</span><span class="token plain"> </span><span class="token string" style="color:#e3116c">"apiKey"</span><span class="token punctuation" style="color:#393A34">:</span><span class="token plain"> api_key</span><span class="token punctuation" style="color:#393A34">}</span><span class="token punctuation" style="color:#393A34">)</span><span class="token plain"></span><br></div><div class="token-line" style="color:#393A34"><span class="token plain">    </span><span class="token keyword" style="color:#00009f">return</span><span class="token plain"> response</span><span class="token punctuation" style="color:#393A34">.</span><span class="token plain">json</span><span class="token punctuation" style="color:#393A34">(</span><span class="token punctuation" style="color:#393A34">)</span><span class="token punctuation" style="color:#393A34">[</span><span class="token string" style="color:#e3116c">"articles"</span><span class="token punctuation" style="color:#393A34">]</span><span class="token plain"></span><br></div><div class="token-line" style="color:#393A34"><span class="token plain" style="display:inline-block"></span><br></div><div class="token-line" style="color:#393A34"><span class="token plain"></span><span class="token keyword" style="color:#00009f">def</span><span class="token plain"> </span><span class="token function" style="color:#d73a49">fetch_stock_price</span><span class="token punctuation" style="color:#393A34">(</span><span class="token plain">api_key</span><span class="token punctuation" style="color:#393A34">,</span><span class="token plain"> symbol</span><span class="token punctuation" style="color:#393A34">)</span><span class="token punctuation" style="color:#393A34">:</span><span class="token plain"></span><br></div><div class="token-line" style="color:#393A34"><span class="token plain">    response </span><span class="token operator" style="color:#393A34">=</span><span class="token plain"> requests</span><span class="token punctuation" style="color:#393A34">.</span><span class="token plain">get</span><span class="token punctuation" style="color:#393A34">(</span><span class="token plain">stock_api_url</span><span class="token punctuation" style="color:#393A34">,</span><span class="token plain"> params</span><span class="token operator" style="color:#393A34">=</span><span class="token punctuation" style="color:#393A34">{</span><span class="token string" style="color:#e3116c">"function"</span><span class="token punctuation" style="color:#393A34">:</span><span class="token plain"> </span><span class="token string" style="color:#e3116c">"TIME_SERIES_INTRADAY"</span><span class="token punctuation" style="color:#393A34">,</span><span class="token plain"> </span><span class="token string" style="color:#e3116c">"symbol"</span><span class="token punctuation" style="color:#393A34">:</span><span class="token plain"> symbol</span><span class="token punctuation" style="color:#393A34">,</span><span class="token plain"> </span><span class="token string" style="color:#e3116c">"interval"</span><span class="token punctuation" style="color:#393A34">:</span><span class="token plain"> </span><span class="token string" style="color:#e3116c">"1min"</span><span class="token punctuation" style="color:#393A34">,</span><span class="token plain"> </span><span class="token string" style="color:#e3116c">"apikey"</span><span class="token punctuation" style="color:#393A34">:</span><span class="token plain"> api_key</span><span class="token punctuation" style="color:#393A34">}</span><span class="token punctuation" style="color:#393A34">)</span><span class="token plain"></span><br></div><div class="token-line" style="color:#393A34"><span class="token plain">    </span><span class="token keyword" style="color:#00009f">return</span><span class="token plain"> response</span><span class="token punctuation" style="color:#393A34">.</span><span class="token plain">json</span><span class="token punctuation" style="color:#393A34">(</span><span class="token punctuation" style="color:#393A34">)</span><span class="token punctuation" style="color:#393A34">[</span><span class="token string" style="color:#e3116c">"Time Series (1min)"</span><span class="token punctuation" style="color:#393A34">]</span><span class="token plain"></span><br></div><div class="token-line" style="color:#393A34"><span class="token plain" style="display:inline-block"></span><br></div><div class="token-line" style="color:#393A34"><span class="token plain"></span><span class="token comment" style="color:#999988;font-style:italic"># Index news articles in Elasticsearch</span><span class="token plain"></span><br></div><div class="token-line" style="color:#393A34"><span class="token plain"></span><span class="token keyword" style="color:#00009f">def</span><span class="token plain"> </span><span class="token function" style="color:#d73a49">index_news_articles</span><span class="token punctuation" style="color:#393A34">(</span><span class="token plain">articles</span><span class="token punctuation" style="color:#393A34">)</span><span class="token punctuation" style="color:#393A34">:</span><span class="token plain"></span><br></div><div class="token-line" style="color:#393A34"><span class="token plain">    </span><span class="token keyword" style="color:#00009f">for</span><span class="token plain"> article </span><span class="token keyword" style="color:#00009f">in</span><span class="token plain"> articles</span><span class="token punctuation" style="color:#393A34">:</span><span class="token plain"></span><br></div><div class="token-line" style="color:#393A34"><span class="token plain">        es</span><span class="token punctuation" style="color:#393A34">.</span><span class="token plain">index</span><span class="token punctuation" style="color:#393A34">(</span><span class="token plain">index</span><span class="token operator" style="color:#393A34">=</span><span class="token string" style="color:#e3116c">"news"</span><span class="token punctuation" style="color:#393A34">,</span><span class="token plain"> document</span><span class="token operator" style="color:#393A34">=</span><span class="token plain">article</span><span class="token punctuation" style="color:#393A34">)</span><span class="token plain"></span><br></div><div class="token-line" style="color:#393A34"><span class="token plain" style="display:inline-block"></span><br></div><div class="token-line" style="color:#393A34"><span class="token plain"></span><span class="token comment" style="color:#999988;font-style:italic"># Retrieve and generate insights using OpenAI</span><span class="token plain"></span><br></div><div class="token-line" style="color:#393A34"><span class="token plain"></span><span class="token keyword" style="color:#00009f">def</span><span class="token plain"> </span><span class="token function" style="color:#d73a49">generate_insights</span><span class="token punctuation" style="color:#393A34">(</span><span class="token plain">query</span><span class="token punctuation" style="color:#393A34">)</span><span class="token punctuation" style="color:#393A34">:</span><span class="token plain"></span><br></div><div class="token-line" style="color:#393A34"><span class="token plain">    results </span><span class="token operator" style="color:#393A34">=</span><span class="token plain"> es</span><span class="token punctuation" style="color:#393A34">.</span><span class="token plain">search</span><span class="token punctuation" style="color:#393A34">(</span><span class="token plain">index</span><span class="token operator" style="color:#393A34">=</span><span class="token string" style="color:#e3116c">"news"</span><span class="token punctuation" style="color:#393A34">,</span><span class="token plain"> query</span><span class="token operator" style="color:#393A34">=</span><span class="token punctuation" style="color:#393A34">{</span><span class="token string" style="color:#e3116c">"match"</span><span class="token punctuation" style="color:#393A34">:</span><span class="token plain"> </span><span class="token punctuation" style="color:#393A34">{</span><span class="token string" style="color:#e3116c">"content"</span><span class="token punctuation" style="color:#393A34">:</span><span class="token plain"> query</span><span class="token punctuation" style="color:#393A34">}</span><span class="token punctuation" style="color:#393A34">}</span><span class="token punctuation" style="color:#393A34">)</span><span class="token plain"></span><br></div><div class="token-line" style="color:#393A34"><span class="token plain">    context </span><span class="token operator" style="color:#393A34">=</span><span class="token plain"> </span><span class="token string" style="color:#e3116c">" "</span><span class="token punctuation" style="color:#393A34">.</span><span class="token plain">join</span><span class="token punctuation" style="color:#393A34">(</span><span class="token punctuation" style="color:#393A34">[</span><span class="token plain">hit</span><span class="token punctuation" style="color:#393A34">[</span><span class="token string" style="color:#e3116c">"_source"</span><span class="token punctuation" style="color:#393A34">]</span><span class="token punctuation" style="color:#393A34">[</span><span class="token string" style="color:#e3116c">"content"</span><span class="token punctuation" style="color:#393A34">]</span><span class="token plain"> </span><span class="token keyword" style="color:#00009f">for</span><span class="token plain"> hit </span><span class="token keyword" style="color:#00009f">in</span><span class="token plain"> results</span><span class="token punctuation" style="color:#393A34">[</span><span class="token string" style="color:#e3116c">"hits"</span><span class="token punctuation" style="color:#393A34">]</span><span class="token punctuation" style="color:#393A34">[</span><span class="token string" style="color:#e3116c">"hits"</span><span class="token punctuation" style="color:#393A34">]</span><span class="token punctuation" style="color:#393A34">]</span><span class="token punctuation" style="color:#393A34">)</span><span class="token plain"></span><br></div><div class="token-line" style="color:#393A34"><span class="token plain">    response </span><span class="token operator" style="color:#393A34">=</span><span class="token plain"> openai</span><span class="token punctuation" style="color:#393A34">.</span><span class="token plain">Completion</span><span class="token punctuation" style="color:#393A34">.</span><span class="token plain">create</span><span class="token punctuation" style="color:#393A34">(</span><span class="token plain">engine</span><span class="token operator" style="color:#393A34">=</span><span class="token string" style="color:#e3116c">"text-davinci-003"</span><span class="token punctuation" style="color:#393A34">,</span><span class="token plain"> prompt</span><span class="token operator" style="color:#393A34">=</span><span class="token string-interpolation string" style="color:#e3116c">f"Analyze the following news and provide trading insights: </span><span class="token string-interpolation interpolation punctuation" style="color:#393A34">{</span><span class="token string-interpolation interpolation">context</span><span class="token string-interpolation interpolation punctuation" style="color:#393A34">}</span><span class="token string-interpolation string" style="color:#e3116c">"</span><span class="token punctuation" style="color:#393A34">,</span><span class="token plain"> max_tokens</span><span class="token operator" style="color:#393A34">=</span><span class="token number" style="color:#36acaa">150</span><span class="token punctuation" style="color:#393A34">)</span><span class="token plain"></span><br></div><div class="token-line" style="color:#393A34"><span class="token plain">    </span><span class="token keyword" style="color:#00009f">return</span><span class="token plain"> response</span><span class="token punctuation" style="color:#393A34">.</span><span class="token plain">choices</span><span class="token punctuation" style="color:#393A34">[</span><span class="token number" style="color:#36acaa">0</span><span class="token punctuation" style="color:#393A34">]</span><span class="token punctuation" style="color:#393A34">.</span><span class="token plain">text</span><span class="token punctuation" style="color:#393A34">.</span><span class="token plain">strip</span><span class="token punctuation" style="color:#393A34">(</span><span class="token punctuation" style="color:#393A34">)</span><span class="token plain"></span><br></div><div class="token-line" style="color:#393A34"><span class="token plain" style="display:inline-block"></span><br></div><div class="token-line" style="color:#393A34"><span class="token plain"></span><span class="token comment" style="color:#999988;font-style:italic"># Execute trade</span><span class="token plain"></span><br></div><div class="token-line" style="color:#393A34"><span class="token plain"></span><span class="token keyword" style="color:#00009f">def</span><span class="token plain"> </span><span class="token function" style="color:#d73a49">execute_trade</span><span class="token punctuation" style="color:#393A34">(</span><span class="token plain">api_key</span><span class="token punctuation" style="color:#393A34">,</span><span class="token plain"> symbol</span><span class="token punctuation" style="color:#393A34">,</span><span class="token plain"> action</span><span class="token punctuation" style="color:#393A34">,</span><span class="token plain"> quantity</span><span class="token punctuation" style="color:#393A34">)</span><span class="token punctuation" style="color:#393A34">:</span><span class="token plain"></span><br></div><div class="token-line" style="color:#393A34"><span class="token plain">    response </span><span class="token operator" style="color:#393A34">=</span><span class="token plain"> requests</span><span class="token punctuation" style="color:#393A34">.</span><span class="token plain">post</span><span class="token punctuation" style="color:#393A34">(</span><span class="token plain">broker_api_url</span><span class="token punctuation" style="color:#393A34">,</span><span class="token plain"> json</span><span class="token operator" style="color:#393A34">=</span><span class="token punctuation" style="color:#393A34">{</span><span class="token string" style="color:#e3116c">"symbol"</span><span class="token punctuation" style="color:#393A34">:</span><span class="token plain"> symbol</span><span class="token punctuation" style="color:#393A34">,</span><span class="token plain"> </span><span class="token string" style="color:#e3116c">"action"</span><span class="token punctuation" style="color:#393A34">:</span><span class="token plain"> action</span><span class="token punctuation" style="color:#393A34">,</span><span class="token plain"> </span><span class="token string" style="color:#e3116c">"quantity"</span><span class="token punctuation" style="color:#393A34">:</span><span class="token plain"> quantity</span><span class="token punctuation" style="color:#393A34">}</span><span class="token punctuation" style="color:#393A34">,</span><span class="token plain"> headers</span><span class="token operator" style="color:#393A34">=</span><span class="token punctuation" style="color:#393A34">{</span><span class="token string" style="color:#e3116c">"Authorization"</span><span class="token punctuation" style="color:#393A34">:</span><span class="token plain"> </span><span class="token string-interpolation string" style="color:#e3116c">f"Bearer </span><span class="token string-interpolation interpolation punctuation" style="color:#393A34">{</span><span class="token string-interpolation interpolation">api_key</span><span class="token string-interpolation interpolation punctuation" style="color:#393A34">}</span><span class="token string-interpolation string" style="color:#e3116c">"</span><span class="token punctuation" style="color:#393A34">}</span><span class="token punctuation" style="color:#393A34">)</span><span class="token plain"></span><br></div><div class="token-line" style="color:#393A34"><span class="token plain">    </span><span class="token keyword" style="color:#00009f">return</span><span class="token plain"> response</span><span class="token punctuation" style="color:#393A34">.</span><span class="token plain">json</span><span class="token punctuation" style="color:#393A34">(</span><span class="token punctuation" style="color:#393A34">)</span><span class="token plain"></span><br></div><div class="token-line" style="color:#393A34"><span class="token plain" style="display:inline-block"></span><br></div><div class="token-line" style="color:#393A34"><span class="token plain"></span><span class="token comment" style="color:#999988;font-style:italic"># Main trading loop</span><span class="token plain"></span><br></div><div class="token-line" style="color:#393A34"><span class="token plain"></span><span class="token keyword" style="color:#00009f">def</span><span class="token plain"> </span><span class="token function" style="color:#d73a49">trading_loop</span><span class="token punctuation" style="color:#393A34">(</span><span class="token punctuation" style="color:#393A34">)</span><span class="token punctuation" style="color:#393A34">:</span><span class="token plain"></span><br></div><div class="token-line" style="color:#393A34"><span class="token plain">    news_api_key </span><span class="token operator" style="color:#393A34">=</span><span class="token plain"> </span><span class="token string" style="color:#e3116c">"YOUR_NEWS_API_KEY"</span><span class="token plain"></span><br></div><div class="token-line" style="color:#393A34"><span class="token plain">    stock_api_key </span><span class="token operator" style="color:#393A34">=</span><span class="token plain"> </span><span class="token string" style="color:#e3116c">"YOUR_STOCK_API_KEY"</span><span class="token plain"></span><br></div><div class="token-line" style="color:#393A34"><span class="token plain">    broker_api_key </span><span class="token operator" style="color:#393A34">=</span><span class="token plain"> </span><span class="token string" style="color:#e3116c">"YOUR_BROKER_API_KEY"</span><span class="token plain"></span><br></div><div class="token-line" style="color:#393A34"><span class="token plain">    query </span><span class="token operator" style="color:#393A34">=</span><span class="token plain"> </span><span class="token string" style="color:#e3116c">"impact of current events on tech stocks"</span><span class="token plain"></span><br></div><div class="token-line" style="color:#393A34"><span class="token plain" style="display:inline-block"></span><br></div><div class="token-line" style="color:#393A34"><span class="token plain">    </span><span class="token comment" style="color:#999988;font-style:italic"># Fetch and index news articles</span><span class="token plain"></span><br></div><div class="token-line" style="color:#393A34"><span class="token plain">    articles </span><span class="token operator" style="color:#393A34">=</span><span class="token plain"> fetch_news</span><span class="token punctuation" style="color:#393A34">(</span><span class="token plain">news_api_key</span><span class="token punctuation" style="color:#393A34">,</span><span class="token plain"> query</span><span class="token punctuation" style="color:#393A34">)</span><span class="token plain"></span><br></div><div class="token-line" style="color:#393A34"><span class="token plain">    index_news_articles</span><span class="token punctuation" style="color:#393A34">(</span><span class="token plain">articles</span><span class="token punctuation" style="color:#393A34">)</span><span class="token plain"></span><br></div><div class="token-line" style="color:#393A34"><span class="token plain" style="display:inline-block"></span><br></div><div class="token-line" style="color:#393A34"><span class="token plain">    </span><span class="token comment" style="color:#999988;font-style:italic"># Generate insights</span><span class="token plain"></span><br></div><div class="token-line" style="color:#393A34"><span class="token plain">    insights </span><span class="token operator" style="color:#393A34">=</span><span class="token plain"> generate_insights</span><span class="token punctuation" style="color:#393A34">(</span><span class="token plain">query</span><span class="token punctuation" style="color:#393A34">)</span><span class="token plain"></span><br></div><div class="token-line" style="color:#393A34"><span class="token plain">    </span><span class="token keyword" style="color:#00009f">print</span><span class="token punctuation" style="color:#393A34">(</span><span class="token string-interpolation string" style="color:#e3116c">f"Generated Insights: </span><span class="token string-interpolation interpolation punctuation" style="color:#393A34">{</span><span class="token string-interpolation interpolation">insights</span><span class="token string-interpolation interpolation punctuation" style="color:#393A34">}</span><span class="token string-interpolation string" style="color:#e3116c">"</span><span class="token punctuation" style="color:#393A34">)</span><span class="token plain"></span><br></div><div class="token-line" style="color:#393A34"><span class="token plain" style="display:inline-block"></span><br></div><div class="token-line" style="color:#393A34"><span class="token plain">    </span><span class="token comment" style="color:#999988;font-style:italic"># Make trading decision</span><span class="token plain"></span><br></div><div class="token-line" style="color:#393A34"><span class="token plain">    </span><span class="token keyword" style="color:#00009f">if</span><span class="token plain"> </span><span class="token string" style="color:#e3116c">"positive sentiment"</span><span class="token plain"> </span><span class="token keyword" style="color:#00009f">in</span><span class="token plain"> insights</span><span class="token punctuation" style="color:#393A34">:</span><span class="token plain"></span><br></div><div class="token-line" style="color:#393A34"><span class="token plain">        execute_trade</span><span class="token punctuation" style="color:#393A34">(</span><span class="token plain">broker_api_key</span><span class="token punctuation" style="color:#393A34">,</span><span class="token plain"> </span><span class="token string" style="color:#e3116c">"AAPL"</span><span class="token punctuation" style="color:#393A34">,</span><span class="token plain"> </span><span class="token string" style="color:#e3116c">"buy"</span><span class="token punctuation" style="color:#393A34">,</span><span class="token plain"> </span><span class="token number" style="color:#36acaa">10</span><span class="token punctuation" style="color:#393A34">)</span><span class="token plain"></span><br></div><div class="token-line" style="color:#393A34"><span class="token plain">    </span><span class="token keyword" style="color:#00009f">elif</span><span class="token plain"> </span><span class="token string" style="color:#e3116c">"negative sentiment"</span><span class="token plain"> </span><span class="token keyword" style="color:#00009f">in</span><span class="token plain"> insights</span><span class="token punctuation" style="color:#393A34">:</span><span class="token plain"></span><br></div><div class="token-line" style="color:#393A34"><span class="token plain">        execute_trade</span><span class="token punctuation" style="color:#393A34">(</span><span class="token plain">broker_api_key</span><span class="token punctuation" style="color:#393A34">,</span><span class="token plain"> </span><span class="token string" style="color:#e3116c">"AAPL"</span><span class="token punctuation" style="color:#393A34">,</span><span class="token plain"> </span><span class="token string" style="color:#e3116c">"sell"</span><span class="token punctuation" style="color:#393A34">,</span><span class="token plain"> </span><span class="token number" style="color:#36acaa">10</span><span class="token punctuation" style="color:#393A34">)</span><span class="token plain"></span><br></div><div class="token-line" style="color:#393A34"><span class="token plain" style="display:inline-block"></span><br></div><div class="token-line" style="color:#393A34"><span class="token plain"></span><span class="token comment" style="color:#999988;font-style:italic"># Run the trading loop</span><span class="token plain"></span><br></div><div class="token-line" style="color:#393A34"><span class="token plain">trading_loop</span><span class="token punctuation" style="color:#393A34">(</span><span class="token punctuation" style="color:#393A34">)</span><br></div></code></pre></div></div>
<h2 class="anchor anchorTargetHideOnScrollNavbar_vjPI" id="considerations">Considerations<a href="https://dheerajinampudi.com/blog/spaces/ai-space/rag-in-trading#considerations" class="hash-link" aria-label="Direct link to Considerations" title="Direct link to Considerations" translate="no">​</a></h2>
<ul>
<li class=""><strong>Data Latency:</strong> Ensure that the data retrieval and processing pipeline is optimized for minimal latency to make timely trading decisions.</li>
<li class=""><strong>Model Accuracy:</strong> Fine-tune your RAG model on financial data to improve the accuracy of generated insights.</li>
<li class=""><strong>Risk Management:</strong> Incorporate risk management strategies in your trading algorithms to mitigate potential losses.</li>
<li class=""><strong>Regulatory Compliance:</strong> Ensure that your trading activities comply with relevant financial regulations and laws.</li>
</ul>
<h2 class="anchor anchorTargetHideOnScrollNavbar_vjPI" id="takeaway">Takeaway<a href="https://dheerajinampudi.com/blog/spaces/ai-space/rag-in-trading#takeaway" class="hash-link" aria-label="Direct link to Takeaway" title="Direct link to Takeaway" translate="no">​</a></h2>
<p>Using RAG in trading can provide a significant edge by leveraging real-time data and advanced AI models to make informed decisions.</p>]]></content:encoded>
            <category>Spaces</category>
            <category>AI Space</category>
            <category>RAG</category>
        </item>
        <item>
            <title><![CDATA[Streamlit apps]]></title>
            <link>https://dheerajinampudi.com/blog/spaces/apps/streamlit-apps</link>
            <guid>https://dheerajinampudi.com/blog/spaces/apps/streamlit-apps</guid>
            <pubDate>Sat, 03 Aug 2024 00:00:00 GMT</pubDate>
            <description><![CDATA[Small Streamlit apps I built: an AI answer evaluator and a house-vs-market ROI calculator.]]></description>
            <content:encoded><![CDATA[<h2 class="anchor anchorTargetHideOnScrollNavbar_vjPI" id="ai-answer-evaluator">AI Answer Evaluator<a href="https://dheerajinampudi.com/blog/spaces/apps/streamlit-apps#ai-answer-evaluator" class="hash-link" aria-label="Direct link to AI Answer Evaluator" title="Direct link to AI Answer Evaluator" translate="no">​</a></h2>
<p><a href="https://llm-judge.streamlit.app/" target="_blank" rel="noopener noreferrer" class="">AI Answer Evaluator</a></p>
<p>The AI Answer Evaluator is a streamlined tool designed to assess the quality of AI-generated answers by comparing them against expected answers. It uses LLM  to identify factual discrepancies, missing information, and errors, and provides a detailed evaluation with a score out of 10.s</p>
<p>The app also includes a customizable evaluation prompt template and offers insights into token usage, including cost estimation. Perfect for developers, researchers, and educators, this app helps analyze and refine AI outputs effectively while ensuring clarity and precision in results.</p>
<h2 class="anchor anchorTargetHideOnScrollNavbar_vjPI" id="house-vs-market-roi-app">House vs Market ROI App<a href="https://dheerajinampudi.com/blog/spaces/apps/streamlit-apps#house-vs-market-roi-app" class="hash-link" aria-label="Direct link to House vs Market ROI App" title="Direct link to House vs Market ROI App" translate="no">​</a></h2>
<p><a href="https://houseroi.streamlit.app/" target="_blank" rel="noopener noreferrer" class="">House vs Market ROI App</a></p>
<p>This is an app about comparing the return on investment (ROI) of buying a flat and investing in the market. It discusses buying a flat, living in it for ten years, and then selling it.</p>]]></content:encoded>
            <category>Spaces</category>
            <category>Apps</category>
        </item>
        <item>
            <title><![CDATA[RAG intro]]></title>
            <link>https://dheerajinampudi.com/blog/spaces/ai-space/rag-intro</link>
            <guid>https://dheerajinampudi.com/blog/spaces/ai-space/rag-intro</guid>
            <pubDate>Tue, 12 Dec 2023 00:00:00 GMT</pubDate>
            <description><![CDATA[What retrieval-augmented generation is, explained through its three basic steps of retrieval, augmentation and generation.]]></description>
            <content:encoded><![CDATA[<p>Ever wondered how some AI systems seem to know exactly what you need? They might be using a technique called retrieval-augmented generation (RAG). It's like giving the AI a little research assistant:</p>
<ol>
<li class="">When you ask a question, the AI first looks for relevant information in its own knowledge base or in external sources.</li>
<li class="">It combines what it found with your question to create a detailed prompt.</li>
<li class="">It uses a powerful language model to generate a response that's both informed and tailored to your needs.</li>
</ol>
<h2 class="anchor anchorTargetHideOnScrollNavbar_vjPI" id="rag-definition">RAG definition<a href="https://dheerajinampudi.com/blog/spaces/ai-space/rag-intro#rag-definition" class="hash-link" aria-label="Direct link to RAG definition" title="Direct link to RAG definition" translate="no">​</a></h2>
<p>In its most basic form, a RAG application runs three steps:</p>
<ul>
<li class=""><strong>Retrieval:</strong> The user's request is used to query an outside source of information. This might mean querying a vector store, running a keyword search over some text, or querying a SQL database. The goal of the retrieval step is to obtain supporting data that helps the LLM provide a useful response.</li>
<li class=""><strong>Augmentation:</strong> The supporting data from the retrieval step is combined with the user's request, often using a template with additional formatting and instructions for the LLM, to create a prompt.</li>
<li class=""><strong>Generation:</strong> The resulting prompt is passed to the LLM, which generates a response to the user's request.</li>
</ul>
<div class="language-text codeBlockContainer_Ckt0 theme-code-block" style="--prism-color:#393A34;--prism-background-color:#f6f8fa"><div class="codeBlockContent_QJqH"><pre tabindex="0" class="prism-code language-text codeBlock_bY9V thin-scrollbar" style="color:#393A34;background-color:#f6f8fa"><code class="codeBlockLines_e6Vv"><div class="token-line" style="color:#393A34"><span class="token plain">User request ──► Retrieval ──► Augmentation (request + supporting data) ──► Generation ──► Response</span><br></div></code></pre></div></div>]]></content:encoded>
            <category>Spaces</category>
            <category>AI Space</category>
            <category>RAG</category>
        </item>
        <item>
            <title><![CDATA[Founder resources]]></title>
            <link>https://dheerajinampudi.com/blog/spaces/founder-mindset/founder-resources</link>
            <guid>https://dheerajinampudi.com/blog/spaces/founder-mindset/founder-resources</guid>
            <pubDate>Tue, 12 Dec 2023 00:00:00 GMT</pubDate>
            <description><![CDATA[Founder blogs, field guides and short videos I keep coming back to.]]></description>
            <content:encoded><![CDATA[<ul>
<li class="">Posthog Founders [<a href="https://posthog.com/founders/cracked-manifesto" target="_blank" rel="noopener noreferrer" class="">https://posthog.com/founders/cracked-manifesto</a>]</li>
<li class="">Basecamp Shape Up [<a href="https://basecamp.com/shapeup" target="_blank" rel="noopener noreferrer" class="">https://basecamp.com/shapeup</a>]</li>
<li class="">Unusual Ventures Field Guide [<a href="https://www.unusual.vc/topics/starting-your-company" target="_blank" rel="noopener noreferrer" class="">https://www.unusual.vc/topics/starting-your-company</a>]</li>
</ul>
<h2 class="anchor anchorTargetHideOnScrollNavbar_vjPI" id="youtube-shorts">Youtube shorts<a href="https://dheerajinampudi.com/blog/spaces/founder-mindset/founder-resources#youtube-shorts" class="hash-link" aria-label="Direct link to Youtube shorts" title="Direct link to Youtube shorts" translate="no">​</a></h2>
<ul>
<li class="">Determination [<a href="https://www.youtube.com/shorts/tmWd5rk3Kwc" target="_blank" rel="noopener noreferrer" class="">https://www.youtube.com/shorts/tmWd5rk3Kwc</a>]</li>
<li class="">Surround by right people [<a href="https://www.youtube.com/shorts/IaUDpM3j6rA" target="_blank" rel="noopener noreferrer" class="">https://www.youtube.com/shorts/IaUDpM3j6rA</a>]</li>
</ul>]]></content:encoded>
            <category>Spaces</category>
            <category>Founder Mindset</category>
        </item>
        <item>
            <title><![CDATA[Sam Altman's talk for founders: it's all about grit]]></title>
            <link>https://dheerajinampudi.com/blog/spaces/founder-mindset/sam-altman-on-grit</link>
            <guid>https://dheerajinampudi.com/blog/spaces/founder-mindset/sam-altman-on-grit</guid>
            <pubDate>Tue, 12 Dec 2023 00:00:00 GMT</pubDate>
            <description><![CDATA[My notes from Sam Altman's talk for founders: grit, focus, self-belief and real relationships.]]></description>
            <content:encoded><![CDATA[<p><img decoding="async" loading="lazy" alt="alt text" src="https://dheerajinampudi.com/assets/images/image-987ef743dadc343097e2ad7b9a4ff3d5.png" width="1712" height="834" class="img_ev3q"></p>
<h2 class="anchor anchorTargetHideOnScrollNavbar_vjPI" id="forget-the-fancy-stuff-its-all-about-grit">Forget the Fancy Stuff, It's All About Grit<a href="https://dheerajinampudi.com/blog/spaces/founder-mindset/sam-altman-on-grit#forget-the-fancy-stuff-its-all-about-grit" class="hash-link" aria-label="Direct link to Forget the Fancy Stuff, It's All About Grit" title="Direct link to Forget the Fancy Stuff, It's All About Grit" translate="no">​</a></h2>
<p>Yeah, brains, experience, and connections are great. But Altman says the real secret sauce is something else: <strong>never giving up</strong>.</p>
<p>He's seen it time and time again. The founders who win aren't always the smartest ones in the room. They're the ones who keep going when things get tough. The ones who say, "I'll figure this out" even when it seems impossible.</p>
<p>The best part? You don't have to be born with this grit. Altman believes you can learn it, just like any other skill.</p>
<h2 class="anchor anchorTargetHideOnScrollNavbar_vjPI" id="three-things-to-focus-on">Three Things to Focus On<a href="https://dheerajinampudi.com/blog/spaces/founder-mindset/sam-altman-on-grit#three-things-to-focus-on" class="hash-link" aria-label="Direct link to Three Things to Focus On" title="Direct link to Three Things to Focus On" translate="no">​</a></h2>
<p>Altman breaks it down into three things you need to nail:</p>
<ol>
<li class=""><strong>Laser Focus:</strong>  Don't get distracted by shiny objects. Pick your goal and go after it with everything you've got.</li>
<li class=""><strong>Believe in Yourself:</strong> This isn't just about positive thinking. You need to <em>really</em> believe your idea can change the world. That confidence is contagious.</li>
<li class=""><strong>Build Real Relationships:</strong>  Founding a company is a team sport. You need to connect with people, whether it's your team, your customers, or your investors.</li>
</ol>
<h2 class="anchor anchorTargetHideOnScrollNavbar_vjPI" id="keep-it-simple-stupid">Keep It Simple, Stupid<a href="https://dheerajinampudi.com/blog/spaces/founder-mindset/sam-altman-on-grit#keep-it-simple-stupid" class="hash-link" aria-label="Direct link to Keep It Simple, Stupid" title="Direct link to Keep It Simple, Stupid" translate="no">​</a></h2>
<p>Altman has another gem: If you can't explain your idea in 25 words or less, you're in trouble. The best founders can pitch their vision clearly and concisely.</p>
<h2 class="anchor anchorTargetHideOnScrollNavbar_vjPI" id="aim-high-its-actually-easier">Aim High, It's Actually Easier<a href="https://dheerajinampudi.com/blog/spaces/founder-mindset/sam-altman-on-grit#aim-high-its-actually-easier" class="hash-link" aria-label="Direct link to Aim High, It's Actually Easier" title="Direct link to Aim High, It's Actually Easier" translate="no">​</a></h2>
<p>This might sound crazy, but Altman says it's <em>easier</em> to start a hard startup than an easy one. Why? Because a bold vision attracts passionate people. No one wants to work on something boring.</p>
<h2 class="anchor anchorTargetHideOnScrollNavbar_vjPI" id="bottom-line">Bottom Line<a href="https://dheerajinampudi.com/blog/spaces/founder-mindset/sam-altman-on-grit#bottom-line" class="hash-link" aria-label="Direct link to Bottom Line" title="Direct link to Bottom Line" translate="no">​</a></h2>
<p>Being a founder is hard work, no doubt about it. But if you have the grit, the focus, and a clear vision, you've got a fighting chance.</p>
<p>So, go out there and build something amazing!</p>]]></content:encoded>
            <category>Spaces</category>
            <category>Founder Mindset</category>
        </item>
        <item>
            <title><![CDATA[Graph triples]]></title>
            <link>https://dheerajinampudi.com/blog/spaces/ai-space/graph-triples</link>
            <guid>https://dheerajinampudi.com/blog/spaces/ai-space/graph-triples</guid>
            <pubDate>Sun, 10 Sep 2023 00:00:00 GMT</pubDate>
            <description><![CDATA[How subject–predicate–object triples form the building blocks of RDF knowledge graphs, with Turtle examples and SPARQL querying.]]></description>
            <content:encoded><![CDATA[<p>Triples are a fundamental concept in knowledge graphs, especially those based on the Resource Description Framework (RDF).</p>
<h2 class="anchor anchorTargetHideOnScrollNavbar_vjPI" id="what-a-triple-is">What a triple is<a href="https://dheerajinampudi.com/blog/spaces/ai-space/graph-triples#what-a-triple-is" class="hash-link" aria-label="Direct link to What a triple is" title="Direct link to What a triple is" translate="no">​</a></h2>
<p>A triple is a data structure with three components:</p>
<ol>
<li class=""><strong>Subject:</strong> the entity or resource being described.</li>
<li class=""><strong>Predicate:</strong> the attribute or relationship of the subject.</li>
<li class=""><strong>Object:</strong> the value, or another entity, that the subject is related to.</li>
</ol>
<p>Together, these three parts form a statement about the data, often likened to a simple sentence with a subject, verb and object. For example:</p>
<table><thead><tr><th>Subject</th><th>Predicate</th><th>Object</th></tr></thead><tbody><tr><td>"Alice"</td><td>"knows"</td><td>"Bob"</td></tr></tbody></table>
<p>This triple can be represented as:</p>
<div class="language-text codeBlockContainer_Ckt0 theme-code-block" style="--prism-color:#393A34;--prism-background-color:#f6f8fa"><div class="codeBlockContent_QJqH"><pre tabindex="0" class="prism-code language-text codeBlock_bY9V thin-scrollbar" style="color:#393A34;background-color:#f6f8fa"><code class="codeBlockLines_e6Vv"><div class="token-line" style="color:#393A34"><span class="token plain">&lt;Alice&gt; &lt;knows&gt; &lt;Bob&gt;</span><br></div></code></pre></div></div>
<h2 class="anchor anchorTargetHideOnScrollNavbar_vjPI" id="triples-in-rdf">Triples in RDF<a href="https://dheerajinampudi.com/blog/spaces/ai-space/graph-triples#triples-in-rdf" class="hash-link" aria-label="Direct link to Triples in RDF" title="Direct link to Triples in RDF" translate="no">​</a></h2>
<p>In RDF, triples make assertions about resources as subject–predicate–object expressions. They can be stored in an RDF graph, where each subject and object is a node and each predicate is an edge connecting two nodes.</p>
<h3 class="anchor anchorTargetHideOnScrollNavbar_vjPI" id="example-triples-in-rdf-format">Example triples in RDF format<a href="https://dheerajinampudi.com/blog/spaces/ai-space/graph-triples#example-triples-in-rdf-format" class="hash-link" aria-label="Direct link to Example triples in RDF format" title="Direct link to Example triples in RDF format" translate="no">​</a></h3>
<div class="language-turtle codeBlockContainer_Ckt0 theme-code-block" style="--prism-color:#393A34;--prism-background-color:#f6f8fa"><div class="codeBlockContent_QJqH"><pre tabindex="0" class="prism-code language-turtle codeBlock_bY9V thin-scrollbar" style="color:#393A34;background-color:#f6f8fa"><code class="codeBlockLines_e6Vv"><div class="token-line" style="color:#393A34"><span class="token plain">&lt;http://example.org/Alice&gt; &lt;http://example.org/knows&gt; &lt;http://example.org/Bob&gt; .</span><br></div><div class="token-line" style="color:#393A34"><span class="token plain">&lt;http://example.org/Alice&gt; &lt;http://example.org/age&gt; "30" .</span><br></div><div class="token-line" style="color:#393A34"><span class="token plain">&lt;http://example.org/Bob&gt; &lt;http://example.org/age&gt; "25" .</span><br></div></code></pre></div></div>
<p>In this example:</p>
<ul>
<li class="">The first triple states that Alice knows Bob.</li>
<li class="">The second triple states that Alice's age is 30.</li>
<li class="">The third triple states that Bob's age is 25.</li>
</ul>
<h2 class="anchor anchorTargetHideOnScrollNavbar_vjPI" id="usage-in-knowledge-graphs">Usage in knowledge graphs<a href="https://dheerajinampudi.com/blog/spaces/ai-space/graph-triples#usage-in-knowledge-graphs" class="hash-link" aria-label="Direct link to Usage in knowledge graphs" title="Direct link to Usage in knowledge graphs" translate="no">​</a></h2>
<p>Triples are the building blocks of RDF-based knowledge graphs. They represent complex information in a structured form that machines can easily parse and query. This enables powerful querying and the ability to infer new information from existing triples.</p>
<p>Knowledge graphs often use query languages such as SPARQL to query these triples and extract meaningful information from the interconnected data.</p>
<h2 class="anchor anchorTargetHideOnScrollNavbar_vjPI" id="summary">Summary<a href="https://dheerajinampudi.com/blog/spaces/ai-space/graph-triples#summary" class="hash-link" aria-label="Direct link to Summary" title="Direct link to Summary" translate="no">​</a></h2>
<p>Triples are essential for representing data in RDF and knowledge graphs. They allow a structured, flexible and semantically rich representation of information.</p>]]></content:encoded>
            <category>Spaces</category>
            <category>AI Space</category>
            <category>Knowledge Graphs</category>
        </item>
    </channel>
</rss>