​
Working with Routers

Routers are a powerful feature in Kosmoy Studio that allow you to define how requests are routed to different models. This enables you to create sophisticated routing strategies, such as failover mechanisms and load balancing (future feature), to enhance the reliability and performance of your AI applications.

​
Router Concepts

Kosmoy Studio currently supports the following router type:

  • Failover Router: This router type defines a primary model and a secondary model. If the primary model fails to respond, the router automatically routes the request to the secondary model.

Future Router Types:

  • Load Balancing Router (Coming Soon): This router type will distribute requests across multiple models based on a defined algorithm (e.g., round-robin, weighted round-robin), optimizing resource utilization and improving response times.

​
Creating a Router

To create a router in Kosmoy Studio, follow these general steps. Note that the specific configuration options will vary depending on the chosen router type:

  1. Navigate to Routers: From the Kosmoy Studio home page, click on the “Routers” menu in the left-hand navigation bar.

  2. Add a New Router: Click the ”+ ADD” button located in the upper right corner of the Routers section.

  3. Select Router Type: Choose the desired router type from the dropdown menu.

  4. Configure Router: Follow the specific configuration steps for the selected router type, as detailed in the subsections below.

  5. Name and Describe the Router: Give your router a unique name and an optional description. This will help you identify it later.

  6. Review and Create: Review the router configuration and click “Create” to create the router.

​
Router Creation Workflows

​
Creating a Failover Router

  1. Select Router Type: Choose “Failover” from the router type dropdown.
  2. Configure Router:
    • Primary Model: Select the primary model for this router.
    • Secondary Model: Select the secondary model (fallback) for this router.
    • Default Error Response Message: Define a custom error message to be returned if both the primary and secondary models fail.
  3. Name and Describe: Give your router a unique name and an optional description.
  4. Review and Create: Review the configuration and click “Create”.

​
Creating a Load Balancing Router (Future Feature)

Note: The Load Balancing router is a future feature. The steps below are a conceptual outline.

  1. Select Router Type: Choose “Load Balancing” from the router type dropdown.
  2. Configure Router:
    • Models: Select the models that will participate in load balancing.
    • Algorithm: Choose the load balancing algorithm (e.g., round-robin, weighted round-robin).
    • Weights (if applicable): Assign weights to each model to define their relative share of the load.
  3. Name and Describe: Give your router a unique name and an optional description.
  4. Review and Create: Review the configuration and click “Create”.

​
Router Cards

The Routers section displays each created router as a card. Each card shows:

  • Router Icon: An icon representing the router type (e.g., Failover).
  • Router Name: The name you assigned to the router.
  • Description: The description you provided for the router.
  • Edit Icon (Pencil): Click this icon to update the router’s configuration, name, or description (only if the router is not in use).
  • Delete Icon (Trash Bin): Click this icon to remove the router (only if the router is not in use).

​
Router Usage Restrictions

You cannot edit or delete a router if it is currently referenced by other entities within Kosmoy Studio. This includes being used in:

  • Gateways
  • Any other Kosmoy Studio component that references routers.

Before attempting to edit or delete a router, ensure it is not actively used in any of these areas. You may need to modify or remove the router from those components first.

​
Updating a Router

You can update the configuration of a registered router, including its name, description, and parameters, provided it is not currently referenced by any other component.

  1. Navigate to Routers: From the Kosmoy Studio home page, click on the “Routers” icon in the left-hand navigation bar.
  2. Locate the Router Card: Find the card for the router you want to update.
  3. Click the Edit (Pencil) Icon: This will open the update dialog.
  4. Modify Router Parameters: Update the router’s configuration as needed. This may include changing the primary/secondary models for a Failover router, or the model list/algorithm for a Load Balancing router, in addition to its name and description.
  5. Click “Save”: Save the changes.

If you attempt to edit a router that is currently in use, a warning banner will be displayed at the top of the screen, preventing the modification.

​
Removing a Router

You can remove a registered router if it’s no longer needed. However, you cannot delete a router that is currently referenced by any other component.

  1. Navigate to Routers: From the Kosmoy Studio home page, click on the “Routers” icon in the left-hand navigation bar.
  2. Locate the Router Card: Find the card for the router you want to remove.
  3. Click the Delete (Trash Bin) Icon: This will trigger a confirmation prompt.

If you attempt to delete a router that is currently in use, a modal will appear, preventing the deletion and explaining that the router is in use.

  1. Confirm Deletion: Confirm that you want to delete the router.

Warning: Deleting a router is a permanent action and cannot be undone. Ensure that the router is not being referenced by any other component before proceeding.

​
Using Routers

Routers are used in other parts of Kosmoy Studio to define the logic for routing requests to different models. For example, when creating a Gateway, you can select a Router to handle requests directed to that Gateway.

Note: Although routers are not directly used in Assistants or Coded Apps, they are indirectly utilized through Gateways, which can be associated with Assistants and Coded Apps.

ModelsGuardrails