APOLLO: A Language Model for Agents

Apollo is a groundbreaking language model that enables functional conversational agents by replacing the transformer with a neuro-symbolic architecture. By combining the conversational skills of generative AI with the predictability and actionability of rule-based AI, Apollo allows companies to fine-tune agents with unprecedented levels of accuracy, safety, and performance. Through continuous human feedback, Apollo fine-tunes daily, adapting to the unique needs of any industry, company, and use case.

Note: This write-up does not expand on the proprietary learning and training architecture of the neuro-symbolic base model, developed over the past six years in collaboration with 60,000 human agents.

Transformer-Based LLMs Do Not Make Good Agents

Traditional language models rely on transformer architectures, which excel at pattern recognition and language generation. They are particularly effective for a variety of tasks such as text generation, translation, and summarization. However, when it comes to agentic use cases—situations where the model needs to perform actions, make decisions, or interact with tools—transformer-based LLMs face significant challenges. There are five key problems that limit their effectiveness as agents:

1. Transparency – Opaque Reasoning and Lack of Explainability
   

   Transformer-based LLMs function as black-box models, meaning their internal reasoning processes are opaque and not easily interpretable. This lack of transparency presents several challenges. First, users and developers cannot trace how the model arrives at specific outputs, making it hard to understand or trust its decisions. Second, when the model produces incorrect or inappropriate responses, it’s difficult to identify the root cause or implement targeted fixes. In addition, in industries where explainability is crucial for regulatory compliance or ethical considerations, the opaque nature of transformer-based LLMs is a significant drawback.

2. Controllability – Lack of Precise Control Over Outputs
   

   Transformer-based LLMs operate as “black boxes”, making it difficult to control their outputs precisely. The primary method of influencing their behavior is through the system prompt, where users can insert instructions or guidelines. However, instructions are given in natural language, which the model may interpret unpredictably. There’s no guarantee that the model will consistently follow the guidelines as intended. In addition, there aren’t any enforcement mechanisms– there’s no built-in way to enforce compliance with policies or rules. The model may generate outputs that violate guidelines without any internal checks. As a result, achieving precise control over the model’s behavior is challenging, leading to outputs that may not align with desired policies or objectives.

3. Predictability – Inconsistent and Unreliable Responses
   

   Due to their probabilistic nature, transformer-based LLMs can produce unpredictable and inconsistent outputs. The model may produce irrelevant or inappropriate content, including hallucinations—statements that are plausible-sounding but factually incorrect. This unpredictability poses risks in applications where consistent and reliable responses are critical, such as customer service or decision-making systems, and is amplified when the model uses tools, as agents always require.

4. Tool Use – Difficulty in Effective Tool Integration
   

   Transformer-based LLMs struggle with effective and reliable tool use. While recent developments like function calling enable some level of tool integration, the model often has difficulty using tools correctly and efficiently. In addition, interactions with tools are generated textually, which can lead to formatting errors (e.g., incorrect JSON syntax) and misinterpretations. Lastly, they lack robust mechanisms for error detection and recovery when tool executions fail or produce unexpected results. These limitations make it challenging to use transformer-based LLMs as agents that need to interact with external systems, APIs, or databases in a reliable manner.

5. Fine-Tuning- Challenges in Continuous Learning and Adaptation
   

   Fine-tuning transformer-based LLMs for specific agentic tasks is problematic. Fine-tuning adjusts the entire model, which can be overkill when only specific behaviors need modification. It lacks the granularity to fine-tune sub-components or specific decision pathways. In addition, fine-tuning and tool-use do not go well together, making it nearly impossible to fine-tune an agent on a data set that includes tool activation. Lastly and most importantly, fine-tuning LLMs happens at the “conversation” and not the “interaction” level. The state of each interaction is not structured and therefore the desired behavior in a specific situation cannot be detected. Due to these issues, transformer-based LLMs are not well-suited for dynamic environments where tool-use, continuous learning and adaptation are essential.

A Neuro-Symbolic Architecture is better suited for Agents

Apollo introduces a neuro-symbolic approach, merging neural networks with symbolic reasoning. This hybrid model not only understands and generates language but also structures its reasoning process. It can incorporate explicit rules into its inferences, and receive structured feedback on each reasoning component.

The inference process of each interaction includes both generative (“Neural”) and rule-based (“Symbolic”) elements, enabling the native tool-use, controllability, predictability and continuous fine-tuning capacity agents require. This approach relies on obtaining a structured interaction state, created by collecting sensory data to produce a symbolic, parameterized representation of each interaction.

The neuro-symbolic approach bridges the gap between the flexibility of neural networks and the precision of symbolic logic. By merging neural networks with symbolic reasoning, Apollo offers several key advantages:

• Explainability: Symbolic reasoning provides insight into the model’s decision-making process. Users can trace how the model arrives at specific responses, enhancing trust and accountability.
• Modularity: Components can be modified or replaced without affecting the entire system. This flexibility facilitates updates, customization, and scalability.
• Rule Integration: Explicit rules can be incorporated directly into the reasoning process. This ensures compliance with regulations, policies, and domain-specific requirements.
• Data Efficiency: Symbolic components reduce the need for extensive datasets. By leveraging existing knowledge bases and rules, Apollo performs effectively even with limited training data.

Structured Interaction State: The Foundation of Apollo’s Reasoning

At the heart of Apollo’s capabilities lies the Structured Interaction State. This state is a symbolic, parameterized representation of each interaction, created by collecting and organizing sensory data. By converting unstructured inputs into a structured format, Apollo achieves a higher level of understanding and precision in its responses.

• Human and Machine Readable: The structured representation is readable by both humans and software, facilitating transparency and ease of interpretation.
• Captures Contextual Nuances: It includes intents, entities, sentiments, and contextual information essential for accurate processing.
• Enables Advanced Reasoning: This structured state is the basis upon which rules, policies, and reasoning processes are applied.

Each interaction is represented using hundreds of symbolic elements. These elements capture detailed aspects of the conversation, such as intents, entities, actions, and context. This structured representation unlocks advanced capabilities for feedback collection, evaluation, guidance, and accuracy enhancement.

Comparison Table

Comparison Between Transformer-based and Apollo-based Agents

I. Transparency

Transformer-Based Agents: Opaque, Black-Box Models

Transformer-based agents are often referred to as black-box models because:

• Lack of Transparency: The internal workings and decision-making processes are not easily interpretable or accessible. Users cannot see how the model arrives at a particular response.
• Opaque Reasoning: The model’s reasoning pathways are hidden, making it difficult to understand the factors influencing its outputs.
• Challenges in Debugging: Identifying and correcting errors or biases within the model is challenging due to its opaque nature.
• Regulatory Compliance Issues: In industries where explainability is crucial for compliance, the black-box nature of transformer-based agents poses significant challenges.

Apollo: Transparent, White-Box Model

Apollo is designed as a white-box model, offering full transparency in its operations:

• Transparent Reasoning Processes: Apollo’s neuro-symbolic architecture allows users to trace and understand the decision-making steps the model takes.
• Interpretable Outputs: Each response is generated through a structured reasoning process that is accessible and interpretable by users.
• Ease of Debugging and Improvement: The transparent architecture enables developers to identify, diagnose, and fix issues more efficiently.
• Enhanced Trust and Accountability: Transparency fosters trust among users and stakeholders, as they can see exactly how the model arrives at its conclusions.
• Auditability: Apollo’s decisions can be audited, providing a clear trail of how specific responses were generated. This is essential for compliance with industry regulations and ethical standards.
• Customization and Control: Users can modify and adjust the reasoning components, tailoring the model to their specific needs without affecting the entire system.

II. Controllability

Transformer-Based Agents: Limited Control Through System Prompts

In transformer-based Large Language Models (LLMs), agent instructions are typically inserted into the system prompt:

• “You are a customer support representative for ___.”
• Function calling (defining the tools the model can use)
• Dos and Don’ts (guidelines and restrictions)

This method presents several limitations:

• Unstructured Information: Instructions and guidelines are unstructured, making it difficult for the model to process and apply them effectively.
• Inefficiency and Unpredictability: Processing all instructions during inference leads to inefficiencies and unpredictable outcomes.
• Lack of Control: Because these models function as “black boxes”, lacking transparency and controllability, and often perform poorly when required to act as agents, use tools, or follow preset instructions. In addition, they rely on context windows to comply with a fixed set of guidelines, whether or not they are relevant to the interaction at hand.

Apollo: An Advanced Instruction Environment

Apollo addresses these limitations by introducing an advanced instruction environment called “Instructions.” This environment enables 1,000 times the specifications of a regular system prompt through a structured approach:

• Structured Instructions: All components—including API integrations established during setup, conditional rules, policies, and specific settings for each tool—are organized within a structured framework.
• Fine-grained Control: Apollo allows precise control over its outputs. Users can define specific rules and guidelines that the model strictly adheres to during interactions. This ensures that the AI’s responses align with company policies, industry regulations, and ethical standards.
• Targeted Retrieval: At inference time, Apollo selectively retrieves only the relevant pieces of information from the Instructions that apply to the specific situation. This targeted retrieval enhances efficiency and ensures that responses are contextually appropriate and compliant with all necessary guidelines.

Situational Rules and Guidelines

Apollo utilizes structured entity representations to determine relevant instructions and guidelines for each interaction. By focusing only on necessary rules at any given moment, the model enhances efficiency and reduces the likelihood of errors or irrelevant responses. It can trigger situational rules when a specific conversational situation emerges. For example, if a customer is looking to buy a certain product, a pre-inserted situational rule will be triggered, informing the customer that this product is currently on a special summer sale.

III. Predictability

Transformer-Based Agents: Unpredictable Outcomes

Transformer-based agents predict the next token in a sequence based on probability distributions learned during training. While this enables fluent language generation, it also leads to several issues:

• Unpredictable Behavior: The model may generate inconsistent or unexpected responses, making it difficult to trust the outcomes.
• Lack of Reliability: Small changes in input can lead to vastly different outputs, affecting the consistency of the agent’s performance.
• Inability to Trust the End Result: Because the model’s reasoning process is opaque, it’s challenging to ensure that the generated responses meet the required standards for accuracy and compliance.

Apollo: A Comprehensive, Structured Reasoning Process

Apollo’s reasoning process blends generative AI with rule-based components. Rather than merely predicting the next token, Apollo follows a detailed reasoning process:

1. Understanding Stage: The user’s input is translated into the Structured Interaction State. This involves parsing the natural language input and mapping it to a formal, structured representation.
2. Rule and Policy Cross-Check: Apollo cross-references the Structured Interaction State with predefined rules and policies to ensure compliance with company standards, legal regulations, and ethical guidelines.
3. Action Determination: Based on the analysis, Apollo decides which action to take. This could involve providing an answer, asking for clarification, or initiating a process.
4. Tool Selection and Activation: If necessary, pre-configured tools are selected and activated. Thanks to its structured handling of tool integrations, Apollo can execute multiple tools simultaneously if the situation requires it.
5. Tool Execution and Verification: The results from the tool executions are verified against expected outcomes. This step includes error checking and validation according to predefined success criteria.
6. Fallback Procedures: If any issues arise, fallback mechanisms are triggered according to preset instructions. This ensures that the conversation continues smoothly even when unexpected situations occur.
7. Response Generation: Only after completing all the previous steps does Apollo generate the final text response. This ensures that the output is accurate, contextually appropriate, and compliant with all relevant guidelines.

By integrating symbolic reasoning, Apollo reduces unexpected behaviors common in purely generative models. It produces consistent and reliable outputs, enhancing trust and reliability in AI-driven processes.

Tool Use

Transformer-Based Agents: Struggling with Tools, to say the least

Transformer-based agents aren’t good with tools:

• Difficulty with Structured Outputs: They fail to flawlessly adhere to different types of structured outputs at inference time (e.g. JSON calls), leading to formatting errors and failed executions.
• Limited Endpoints: Cannot master multiple endpoints, making it challenging to interact with various APIs or services.
• Lack of Control and Definition: Cannot control or define tool execution reliably, resulting in inconsistent performance.
• High Failure Rates: Frequent failures in executing API calls, leading to ungrounded or incorrect answers, making long sequence multi-turn conversations practically infeasible.

Apollo: Tool-Native with 100% Success in Tool Use

Apollo handles tool activation and processing in a structured manner, rather than a generative one:

• Seamless Activation: The model activates tools based on context and intent without ambiguity.
• Structured Processing: It processes multiple types of tool outputs (e.g. JSON, key:Value, etc.) and integrates them into responses seamlessly.
• Scalability: Supports an unlimited number of tools by adding them to its structured handling framework.
• Reliability: With a 100% success rate in tool activation, Apollo eliminates failures to execute API calls and always returns grounded answers.
• Tool-Specific Rules and Guidelines: Supports detailed, tool-specific rules and guidelines, enhancing functionality and user experience.
• Groundness: The ability to use tools masterfully enables Apollo to return grounded answers, rooted in any number of categorized sources and resulting in higher accuracy compared with relying on the training data.

V. Fine-Tuning

Transformer-Based Agents: Challenges with Fine-Tuning

Transformer-based agents cannot be fine-tuned effectively:

• Static Behavior: While LLMs can be fine-tuned, LLM agents remain static and cannot adapt based on specific interactions or feedback.
• Lack of Granularity: It is impossible to fine-tune sub-components or specific behaviors within the agent.
• Infeasibility for Agents: Fine-tuning agents for specific tasks is challenging and often impractical.

Apollo: Introducing Continuous Fine-Tuning via Human Feedback

Apollo evolves continuously through human feedback via fine-tuning:

• Sub-Interaction Level Feedback: Instead of providing feedback only at the conversation or interaction level, Apollo enables granular feedback on multiple sub-interaction reasoning elements. Users can target specific parts of the reasoning process for improvement, leading to more precise adjustments and accelerated learning.
• Continuous Fine-Tuning: Incorporates new data and human feedback regularly. By combining traditional fine-tuning with reinforcement learning, the model adapts quickly without the need for large-scale version upgrades.
• Evaluation and Focused Learning: Apollo’s structured elements enable comprehensive evaluations across numerous data points per interaction. This detailed evaluation process identifies areas for improvement with precision, allowing for targeted enhancements in weaker areas while maintaining strengths.
• Continuous Learning: Because Apollo learns directly from human feedback on each interaction, each version is better aligned and improved compared to the previous one, and can be used as the foundation for the next version.
• Incremental Enhancements: Regular updates prevent the stagnation that can occur between major version releases, ensuring the model remains up-to-date with the latest information and user preferences.
• Structured Evaluation: By relying on consistent structuring elements, Apollo unlocks granular evaluations of interactions. This allows the model to evolve through a detailed understanding of data points, enabling targeted improvements where it’s less strong while maintaining its strengths.

The Role of Transformers in Apollo

In Apollo, transformer-based language models are utilized during the inference process, but their role is specialized:

• Natural Language Understanding (NLU): Transformers assist in comprehending user input by parsing and interpreting natural language, identifying intents, entities, and sentiments.
• Natural Language Processing (NLP): They perform linguistic tasks within the inference pipeline, such as tokenization, part-of-speech tagging, and syntactic parsing.
• Natural Language Generation (NLG): Transformers generate the actual message text after Apollo’s structured reasoning process produces the parameters, placeholders, and components that need translation into natural language.

However, transformers in Apollo do not decide what action to take or how to execute it. The decision-making is governed by the structured reasoning process that integrates both generative (“Neural”) and rule-based (“Symbolic”) elements. This combination enables the native tool use, controllability, predictability, and continuous fine-tuning capabilities that agents require.

Ensuring Safety and Accuracy

Combining generative and rule-based approaches minimizes risks associated with AI outputs.

• Content Moderation: Multiple layers of moderation are in place to avoid generating inappropriate or harmful content, some inherent to Apollo, and some can be defined per agent.
• Information Accuracy: Provides consistent and accurate information, reducing misinformation.
• Auditability: The reasoning process can be reviewed and audited, meeting compliance and governance standards.

Empowering Businesses with Apollo

Companies can harness Apollo to develop AI agents tailored to their specific needs.

• Customization: Fine-tune the model to reflect brand voice, policies, and customer service standards.
• Compliance: Ensure all interactions adhere to industry regulations and ethical guidelines.
• Efficiency: Automate complex tasks with high accuracy, reducing operational costs.
• Adaptability: Continuously improve and adapt the model based on real-world interactions and feedback.

Conclusion

Apollo represents a significant advancement in language modeling. Its neuro-symbolic architecture offers a powerful solution for organizations seeking AI systems that are intelligent, controllable, predictable, and continuously improving. By embracing both generative capabilities and rule-based reasoning, Apollo sets a new standard for performance, safety, and accuracy in AI-driven interactions.