Model Node

The model node allows you to send requests to small language models (SLMs) and large language models (LLMs).

Arcee SLMs are the foundation of Orchestra, as they were trained to excel as agents and within AI networks.

Arcee's Models

Out-of-the box, Orchestra comes with the following Arcee SLMs:

• Virtuoso Large

  • Our most powerful and versatile general-purpose model, designed to excel at handling complex and varied tasks across domains. With state-of-the-art performance, it offers unparalleled capability for nuanced understanding, contextual adaptability, and high accuracy. Its scalability and depth make it ideal for enterprises requiring comprehensive AI solutions.

• Virtuoso Medium

  • A versatile and powerful model, capable of handling complex and varied tasks with precision and adaptability across multiple domains. Ideal for dynamic use cases requiring significant computational power.

• Virtuoso Small

  • A streamlined version of Virtuoso, maintaining robust capabilities for handling complex tasks across domains while offering enhanced cost-efficiency and quicker response times.

• Coder

  • A high-performance model tailored for intricate programming tasks, Coder-Large thrives in software development environments. With its focus on efficiency, reliability, and adaptability, it supports developers in crafting, debugging, and refining code for complex systems.

For detailed information on each model and how they're trained to be the most effective models in agentic systems, see model selection.

Bring Your Own

While Arcee highly recommends using the built-in models for optimal performance, there are times when a customer has their own fine-tuned model or has a requirement to use an outside model. Any model compatible with the OpenAI API can be integrated into Orchestra.

Using the Model Node

Model Configuration

In the model node, you can:

1. Select which model you'd like to use

2. Import any variables from previous nodes in the workflow

3. Write your User and System prompts

4. Specify model parameters in Model Settings

Model Node Settings

In model settings you can define the following settings:

• Max Tokens

  • This defines the maximum number of tokens (or parts of a word) that the model can generate. A higher max_tokens means the model can produce longer responses and is useful when the output should be longer and more detailed. A lower max_tokens means the model will be constrained to a shorter response, which is useful to ensure shorter responses.

• Temperature

  • Temperature controls the randomness in output token selection, where lower values make responses more deterministic, while higher values increase creativity and variability. Higher values are useful for creative tasks such as storytelling and brainstorming, while lower values are useful for precise tasks such as coding and analysis.

• Top P

  • Top P is used to limit token selection to a top percentage of potential next tokens. A higher top_p (for example, top_p > 0.8), can include less probable tokens leading to a more diverse and creative output. A lower top_p (for example, top_p < 0.3), narrows selection to only the most likely tokens leading to a more focused and deterministic output.

• Top K

  • Similar to Top P, Top K limits token selection to the top-k most probable tokens. A higher top_k will lead to more creative outputs, while a lower top_k will lead to more precise and deterministic outputs.

• Repetition Penalty

  • Repetition Penalty penalizes repeated use of the same tokens to encourage varied responses and avoid redundancy. Higher values for repetition penalty discourage repetition leading to more diverse output, which is good for creative tasks. Lower values for repetition penalty allow for more repetition, which is good for tasks where creativity isn't a factor such as writing contracts or technical instructions.

Model Response

{
  "variable1": "This can be any text, file, JSON, image, or output from a previous node",
  "variable2": "Multiple variables can be added to integrate data from numerous sources",
  "ad_model": "\"I'm an AI agent designed to help drive innovation within your business. What will you create with Arcee Orchestra?\""
}

Output of the model node is a JSON object which lists any variables passed into the model as well as the model's response.

Model output can be referenced using the name of the specific model node. Given the example above, {{ad_model}} would return "I'm an AI agent designed to help drive innovation within your business. What will you create with Arcee Orchestra?" .