We have a few updates for our LLM Service.
Model Updates
As we prepare for the Fall semester, we’re looking at changes to our largest models. As of now, GLM 5.1 (our previous LTS model) is marked as deprecated, and we will retire it in early September. We’re currently marking GLM 5.2 as our active model, but will likely replace it with GLM 5.3 when the weights are released (depending on the final license).
We’ve also deployed Qwen 3.8 27b, a surprisingly capable smaller model with image support. If there is a particular model that would be useful to your research, please let us know!
A feature we have just added is the concept of model aliases. These aliases can represent a particular capability or use-case, and will track the current best model for the task. Currently, we have the following aliases:
tiger: This alias points to the most powerful model we are currently hosting, useful in coding agents.cub: This model alias points to a smaller agentic capable model.
We hope this alias system will alleviate some of the friction caused by the model changes that are inevitable with limited hardware. If there are other use cases you would like to track with an alias, please let us know.
Batch API
We’ve also introduced a Batch API support over this summer. It is designed to be compatible with the OpenAP Batch API and you can either use the APIs directly (e.g. through OpenAI python SDK as we show in the Quick Start) or through the Batch Web UI. Using the Batch API provides the following benefits:
- Automatic retries: transient errors (rate limits, server errors) are retried with exponential backoff so you do not have to build retry logic
- Queued when unavailable: if a model is temporarily down, your requests are safely queued and processed when the model comes back online
- Adaptive concurrency: the service automatically adjusts concurrency based on backend engine capacity to maintain throughput
For more details, please read our documentation.
OpenAI Model Access
As part of our enterprise agreement with OpenAI, we have credits available for API access to OpenAI models. We now make a portion of these credits available to projects using the RCD LLM Service. Each allocation is given a default allocation of credits, and then can also consume credits from a shared pool.
Most OpenAI models are available. To see the list, go to our models page, and make sure external models are not hidden. The OpenAI models should appear at the bottom of the list.
Requests can be made to the same endpoint as the open weight models, and you can use the RCD LLM Service Batch API to submit requests to OpenAI models. Since the batching happens at our RCD LLM Service proxy level, you do not directly get the OpenAI batch pricing by submitting to our Batch API, however you can receive the equivalent 50% discount by submitting using the flex service tier in each of your batch requests.
There are currently restrictions on certain API features (e.g. web_search, image_generation). If these pose significant limitations to your research, please let us know. We may be able to lift the restrictions.
For more details, please read our documentation.
Scheduler Improvements
There have also been updates to the scheduler, which should keep the models feeling more performant even has load increases by dynamically adjusting the number of requests allowed to the backend based on real time backend metrics. The scheduler still uses a fair-share policy, deciding which requests to send to the backend when it is saturated based on recent usage.
We’ve also introduce a “Nice” configuration option when you create API keys. You can now configure an API key to have a high “Nice” value, which voluntarily deprioritizes and requests using the key. This is useful so that you can use a large Nice value on your batch requests and default low Nice value on your interactive (e.g. coding harness) requests, which would allow your interactive use to have priority over batch requests.
