Category: Announcements

We have a few updates for our LLM Service.

Model Updates

As we prepare for the Fall semester, we’re looking at changes to our largest models. As of now, GLM 5.1 (our previous LTS model) is marked as deprecated, and we will retire it in early September. We’re currently marking GLM 5.2 as our active model, but will likely replace it with GLM 5.3 when the weights are released (depending on the final license).

We’ve also deployed Qwen 3.8 27b, a surprisingly capable smaller model with image support. If there is a particular model that would be useful to your research, please let us know!

A feature we have just added is the concept of model aliases. These aliases can represent a particular capability or use-case, and will track the current best model for the task. Currently, we have the following aliases:

• tiger: This alias points to the most powerful model we are currently hosting, useful in coding agents.
• cub: This model alias points to a smaller agentic capable model.

We hope this alias system will alleviate some of the friction caused by the model changes that are inevitable with limited hardware. If there are other use cases you would like to track with an alias, please let us know.

Batch API

We’ve also introduced a Batch API support over this summer. It is designed to be compatible with the OpenAP Batch API and you can either use the APIs directly (e.g. through OpenAI python SDK as we show in the Quick Start) or through the Batch Web UI. Using the Batch API provides the following benefits:

• Automatic retries: transient errors (rate limits, server errors) are retried with exponential backoff so you do not have to build retry logic
• Queued when unavailable: if a model is temporarily down, your requests are safely queued and processed when the model comes back online
• Adaptive concurrency: the service automatically adjusts concurrency based on backend engine capacity to maintain throughput

For more details, please read our documentation.

OpenAI Model Access

As part of our enterprise agreement with OpenAI, we have credits available for API access to OpenAI models. We now make a portion of these credits available to projects using the RCD LLM Service. Each allocation is given a default allocation of credits, and then can also consume credits from a shared pool.

Most OpenAI models are available. To see the list, go to our models page, and make sure external models are not hidden. The OpenAI models should appear at the bottom of the list.

Requests can be made to the same endpoint as the open weight models, and you can use the RCD LLM Service Batch API to submit requests to OpenAI models. Since the batching happens at our RCD LLM Service proxy level, you do not directly get the OpenAI batch pricing by submitting to our Batch API, however you can receive the equivalent 50% discount by submitting using the flex service tier in each of your batch requests.

There are currently restrictions on certain API features (e.g. web_search, image_generation). If these pose significant limitations to your research, please let us know. We may be able to lift the restrictions.

For more details, please read our documentation.

Scheduler Improvements

There have also been updates to the scheduler, which should keep the models feeling more performant even has load increases by dynamically adjusting the number of requests allowed to the backend based on real time backend metrics. The scheduler still uses a fair-share policy, deciding which requests to send to the backend when it is saturated based on recent usage.

We’ve also introduce a “Nice” configuration option when you create API keys. You can now configure an API key to have a high “Nice” value, which voluntarily deprioritizes and requests using the key. This is useful so that you can use a large Nice value on your batch requests and default low Nice value on your interactive (e.g. coding harness) requests, which would allow your interactive use to have priority over batch requests.

The RCD team has completed its annual summer maintenance for Palmetto 2 and Indigo. All RCD services are now available.

During the maintenance window, the team completed:

• Linux kernel updates to patch recently identified security vulnerabilities
• Minor NVIDIA driver updates
• Minor InfiniBand driver updates
• Upgrades to the Indigo Data Lake
• Updates to Open OnDemand, including the new Palmetto Desktop experience described below

These updates help maintain the security, reliability, and stability of Clemson’s research computing infrastructure as we prepare for the Fall semester.

We appreciate your patience while our team completed this work. If you experience any issues following maintenance, please submit a support ticket so the RCD team can assist.

Changes to $TMPDIR

The behavior of $TMPDIR for jobs running on Palmetto has changed.

Previously, $TMPDIR pointed to a job-specific directory under /local_scratch:

/local_scratch/slurm.$SLURM_JOBID

Following this maintenance, each user receives a namespaced /tmp directory on the compute node, and $TMPDIR now points to that location.

This change does not change the underlying storage characteristics of $TMPDIR. The directory continues to use the local storage on the compute node and is intended for temporary files created during a job. Its contents are still automatically cleaned up after the job ends.

Most applications and scripts that use the $TMPDIR environment variable should continue to work without modification. However, scripts that explicitly reference /local_scratch/slurm.$SLURM_JOBID should be updated to use $TMPDIR instead.

For more information about local scratch storage and $TMPDIR, see the ⁠Palmetto storage documentation.

New Palmetto Desktop

As part of this maintenance, RCD updated Open OnDemand with a new Palmetto Desktop experience.

The New Palmetto Desktop provides an updated way to launch and use a graphical desktop environment on Palmetto through Open OnDemand. Users can access the new desktop from the Open OnDemand interface alongside other interactive applications.

We encourage users who work with graphical applications or prefer a desktop environment on Palmetto to try the New Palmetto Desktop and share any feedback or issues with the RCD team.

During the week of July 26th, several members of Clemson’s Research Computing and Data (RCD) team will be attending the Practice and Experience in Advanced Research Computing (PEARC) 2026 conference in Minneapolis, MN.

While our team is away, please note that response times for user support inquiries may be delayed.

This year, our team has three accepted publications that we will be presenting at the conference. Below are the details of each and links to helpful resources.

The RCD team has scheduled its annual summer maintenance for Palmetto 2 and Indigo before the start of the Fall semester.

Maintenance will begin on Friday, August 7, 2026, at 9:00 AM. During the maintenance window, all RCD services will be unavailable.

The planned maintenance includes:

• Minor operating system upgrades
• Upgrades to the Indigo Data Lake

These updates are necessary to maintain the security, reliability, and stability of the cluster for the Fall semester.

There are no plans to purge scratch storage during this maintenance. However, users should remember that scratch storage is not backed up. Any important data should be stored in home or project storage.

The summer maintenance is scheduled to conclude on Wednesday, August 12, 2026, at 9:00 AM. If maintenance is completed ahead of schedule, RCD will notify users by email as services become available.

We are currently experiencing a chiller outage at Clemson’s datacenter. To help reduce the cooling load and maintain system stability, we have temporarily limited the number of jobs that can run concurrently on the cluster.

As a result, queue wait times will be longer than usual until the issue is resolved.

We apologize for the inconvenience and appreciate your patience while we work through this situation. We will provide updates here as they become available.

The Research Computing and Data (RCD) team is pleased to announce a significant increase in the scratch storage quota on Palmetto 2.

Effective immediately, the default scratch storage allocation has been increased from 5 TiB and 5 million files to 10 TiB and 10 million files per user.

Purge Policy Remains in Effect

This quota increase does not change Palmetto 2’s existing scratch space purge policy. The /scratch filesystem remains a temporary workspace intended for active research and computation, and files that meet the purge criteria remain subject to removal.

Users are responsible for understanding and complying with the scratch purge policy. Attempts to circumvent the policy—including actions intended solely to prevent files from being identified as inactive or eligible for purge—are prohibited. RCD reserves the right to remove data associated with such activity without prior notice in order to maintain the health, performance, and fairness of the shared filesystem.

Researchers should ensure that important data is stored in appropriate long-term storage locations and should not rely on /scratch for archival or permanent storage.

Why the Change?

As research workflows continue to grow in scale and complexity, many users are working with increasingly large datasets, simulation outputs, machine learning training data, and high-throughput analysis pipelines. Based on user feedback and observed usage patterns, RCD has expanded scratch storage capacity to better support these evolving computational needs.

The updated limits provide:

• 2× more scratch storage capacity (10 TiB)
• 2× higher file count allowance (10 million files)
• Greater flexibility for data-intensive workflows
• Reduced need for users to actively manage scratch space during large projects

About Scratch Storage

The /scratch filesystem on Palmetto 2 is designed for temporary, high-performance storage during active computational work. Users should continue to treat scratch space as temporary storage and maintain copies of important data in appropriate long-term storage locations.

While the per-user quota has been increased, users should be aware that scratch storage is a shared resource with finite capacity. The new quota represents the maximum amount of storage and number of files a user may consume, but it does not guarantee that every user can simultaneously utilize their full allocation. Responsible use of scratch space helps ensure that the filesystem remains performant and available for the entire Palmetto community.

Users can monitor their current scratch usage and quota limits at any time using the checkquota command. Additional information about scratch storage, quotas, and purge policies is available in the Palmetto documentation.

As a reminder:

• Scratch storage is intended for active jobs and working datasets.
• Critical research data should be backed up elsewhere.
• Users are encouraged to periodically remove unneeded files and directories.

The Research Computing and Data (RCD) team is excited to announce our Summer 2026 workshop series, available free of charge to all Clemson University students, faculty, and staff.

Our series will cover various high-performance computing, machine learning, and software development topics. Here’s a summary of what we’ll cover:

• Introduction to Linux
• Introduction to Research Computing on Palmetto
• Introduction to Nextflow
• AI Agent Skills and GPTs for Research
• AI Code Assistants for HPC on Palmetto
• Research Computing on Kubernetes
• Inside the Agent: Demystifying LLM Agents in Practice

You can learn more about the details of each workshop, the schedule, and registration links on the upcoming live training sessions page of our documentation site.

We look forward to seeing you this summer and helping you make the most of Clemson’s research computing resources!

During today’s ReDCAT lunch event, Dr. Carl Ehrett shared updates on several new AI capabilities now available through Clemson Research Computing and Data (RCD), including the RCD LLM Service, Codex on Palmetto, and the Palmetto HPC Helper.

Highlights from the presentation included:

• Using the RCD LLM Service API endpoints with research workflows via Jupyter Notebooks
• Chatting with Clemson-hosted AI models through OpenWebUI
• Connecting Clemson-hosted models to coding assistants such as Codex and Claude Code
• Discussion of research reproducibility and long-term access to open-weight AI models
• Demonstration of the Palmetto HPC Helper, a custom GPT grounded in Palmetto documentation

A recording of this presentation is available below.

If the video player does not appear, use this direct link to the recording instead.

To learn more about these services, please see documentation for the RCD LLM Service and Codex on Palmetto. If you have questions, feel free to contact the RCD team.

We’d like to announce the pilot of our new RCD LLM Service. This service provides no-cost API access to leading open-weight models to all Clemson users.

The service provides access through common API formats including:

• OpenAI Compatible Chat Completions
• OpenAI Compatible Embedding
• OpenAI Compatible Responses
• Anthropic Compatible Messages

These APIs are widely supported by many AI tools and libraries, including coding assistants such as OpenAI Codex, Claude Code, and OpenCode; chat and RAG interfaces such as OpenWebUI and AnythingLLM; and software libraries such as Pydantic AI, LangChain, and OpenAI SDKs.

There are no set rate limits or token quotas. Instead the API has a fair-share scheduling mechanism built in so that if the underlying hardware becomes saturated, requests are held before sending to the underlying engine with priority based on recent usage.

Check out our documentation for details. To get started, you will need to request an allocation in a process similar to Palmetto, then you can register API keys.

As documented in our Acceptable Use Guidelines, any use of the service must remain consistent with Clemson University IT policies, Clemson AI guidance, and any sponsor, contract, or regulatory requirements that apply to your work. Non-academic personal and commercial use is prohibited.

We are excited to announce that the Spring 2026 maintenance work was completed successfully.

All RCD services have been restored and are ready for users to access.

The RCD completed the following tasks:

• Critical updates to network and storage infrastructure were completed.
  • These updates have improved performance and stability for all users of the cluster
• Critical Kernel updates to address CVE-2026-3141 (Copy Fail)
• Mitigations to address CVE-2026-43284 and CVE-2026-43500 (Dirty Frag)
• Nvidia driver upgraded to 580 branch (CUDA 13 support)
  • cuda/13.0.2 module added
• Slurm was upgraded to 25.11.5
• Open OnDemand was upgraded to 4.1.5
• GitLab was updated to v18.11.2

We appreciate your patience during the maintenance period and hope that these changes will improve the user experience.

If you have any questions or have encountered post-maintenance issues, please let us know by submitting a support ticket.