How to Configure Foundational Model Settings Using the Model Playgroundpublic

Before you select a foundation model for your agent, you should compare and evaluate various models in the Model Playground for your agent’s use case. The playground provides an interface where you can compare different models, evaluate models, and tweak model settings.

You can open the Model Playground in one of the following ways:

• In the left-hand menu of the DigitalOcean Control Panel, select GenAI Platform. Go to the GenAI Model Library section, click Getting Started for any model and select Test in Playground.

• When creating an agent, in the Select a model section, click Model Playground.

Compare Models

It is a best practice to compare various foundation models and choose the foundation model that best suits your agent’s job. Larger models typically have better quality than smaller models.

On the Model Playground page, click Compare Another Model to open the comparison view. Select a model from the drop-down list. Toggle the Sync Inputs option to ensure that the two models you are comparing respond to the same question. In the text box, enter your question and click the up arrow.

Evaluate the models and if needed, change model settings, as described in the next sections.

Evaluate Models for Performance and Efficiency

You should start evaluating a foundation model by asking some questions specific to your agent’s objective. For more information on best practices for questions, see Best Practices for Prompt Writing.

Enter your question in the Type your message text box. For example, for a model to be used in a travel agent, your question can be What are some popular travel destinations for a summer vacation?.

If you are satisfied with the model’s responses and metrics, click Create Agent.

In case the model responds with too much text or you want more variability in the responses, then you can change some model settings, as described in the next section.

Configure Model Settings

Optimize your agent’s performance by adjusting settings such as Max Tokens, Temperature, Top P, and Agent Instructions to align with its output and needs. Default settings work well, but fine-tuning improves results.

You can change the model settings to shorten the length or add more variation to the responses. The settings you can configure are:

• Max Tokens: Defines the maximum tokens a model processes per input or output. For model-specific details, see the models page.

  Example

  Example Prompt	Response with Max Token Value: 200	Response with Max Token Value: 512
  What is UN?	The United Nations (UN) is an international organization that aims to promote peace, security, and cooperation among its member states.	The United Nations (UN) is an international organization that was established in 1945 to promote peace, security, and cooperation among its member states. The UN has 193 member states and is headquartered in New York City. Its goals include maintaining international peace and security, promoting sustainable development, protecting human rights, delivering humanitarian aid, and upholding international law. The UN is composed of several key bodies, including the General Assembly, the Security Council, the Economic and Social Council, the Secretariat, and the International Court of Justice.

  In the example with maximum number of tokens of 200, the token limit prevents details such as number of members and including explanation of UN’s main goals and subcommittees in the response.

• Temperature: Controls the model’s creativity, specified as a number between 0 and 1. Lower values produce more predictable and conservative responses, while higher values encourage creativity and variation. Values are rounded to the nearest hundredth. For example, if you enter a value of 0.255, the value is rounded to 0.26.

  Example

  Example Prompt	Response with Temperature Value: 0.0	Response with Temperature Value: 0.5	Response with Temperature Value: 0.7	Response with Temperature Value: 1.0
  What color is the sky?	The sky is blue.	The sky is usually blue during the day, but it can appear different colors depending on weather conditions and time of day.	The sky’s color can vary widely. While it’s often perceived as blue, it can also appear pale blue, azure, cyan, or even take on hues of orange, pink, or purple during sunrise or sunset. Clouds, pollution, and atmospheric conditions can further influence its appearance.	Ah, the enigmatic canvas above! Some say it’s an endless ocean of cerulean, others a chameleon of hues. Perhaps it’s a mood ring for Mother Nature, shifting from the softest baby blue to the deepest indigo, with detours through fiery oranges and dreamy purples. Or maybe it’s just a figment of our collective imagination, a shared hallucination we call “sky.” What do you think it looks like right now?

  These examples show how increasing the temperature tends to produce more varied, creative, and sometimes unpredictable responses. At lower temperatures, the output is more focused and conventional, while higher temperatures allow for more diverse and imaginative answers.

• Top P: Defines the cumulative probability threshold for word selection, specified as a number between 0 and 1. Higher values allow for more diverse outputs, while lower values ensure focused and coherent responses. Values are rounded to the nearest hundredth. For example, if you enter a value of 0.255, the value is rounded to 0.26.

  Example

  Example Prompt	Response with Top P Value: 0.1	Response with Top P Value: 0.5	Response with Top P Value: 0.7	Response with Top P Value: 1.0
  What color is the sky?	The sky is blue.	The sky is typically blue, but it can change color depending on weather conditions and time of day.	The sky’s color varies widely. It’s often blue during clear days, but can appear gray when cloudy, and takes on beautiful hues of orange, pink, and purple during sunrise and sunset. Atmospheric conditions like pollution or dust can also affect its appearance.	The sky is a ever-changing tapestry of colors! While we often think of it as blue, it’s really a dynamic canvas that shifts throughout the day. From the soft pastels of dawn to the vibrant azure of noon, and the fiery spectacle of sunset, the sky never ceases to amaze. Sometimes it’s a moody gray, other times a crisp cerulean. And let’s not forget those rare treats like the ethereal green of the aurora borealis or the ominous green tint before a tornado. What’s your favorite sky color?

  These examples show how increasing the top P value allows for more diverse and creative responses. At lower top P values, the output is more focused and conventional, while with higher values, the model responds with a wider range of possible completions and more varied and potentially more imaginative answers.

  Note

  You can change either Temperature or Top P, or both to change the model’s creativity.

• K-Value: Controls the number of tokens to consider when selecting the next word. Higher values increase the number of tokens considered, allowing for more diverse and creative responses.

• Retrieval Method: These options provide agents with additional guidance for retrieving information and generating responses.

  • Rewrite: The agent uses the current input and the prior context of the conversation to refine the user’s input and remove any ambiguity. The agent uses this to generate a shorter and more precise response. This method uses fewer tokens.
  • Step Back: The agent uses the current input and the prior context of the conversation to generate responses with a slightly broader scope than the original input, but the response remains focused on the overall topic. This method uses more tokens.
  • Sub Queries: The agent uses the current input and the prior context of the conversation to generate two to four succinct versions of the original input. It then uses the original input plus the new versions to broaden and improve search effectiveness and generate a response. This method uses more tokens than the rewrite and step-back methods.
  • None: The agent uses only the current input to retrieve information and doesn’t use any prior context. This method uses fewer tokens.

• Agent Instructions: Context that informs the agent about its purpose and the types of information it should and shouldn’t retrieve. You define an agent’s instructions during creation, but you can change them at any time. See our instruction-writing best practices to learn how to write effective instructions.

To change model settings, click the gear icon, adjust the setting slider, and click Update Settings. Evaluate the model responses and continue to iteratively change the model settings. Once you are satisfied that the model responses meet your criteria, click Create Agent.