Kubernetes cluster

The Kubernetes cluster is independent from Chimera and is used to implement cloud services where resources are dynamically allocated as needed, depending on the workload. We use Kubernetes to host various containerized applications running directly from your web browser, including:

• OpenWebUI for interaction with Large Language Models.
• LangFlow for construction of customized agentic workflows.
• LiteLLM for interfacing with Large Language Models across all applications.

Large Language Models are accessed by these applications using the University Gateway routing interface in the backend. In principle, this allows us to offer an arbitrary set of different providers, from those running purely locally on our hardware, to external ones, including eINFRA and commercial providers, while managing the spending caps per user and accounting on the faculty level and split up between user groups and individual users as needed at the given faculty.

Hardware

The cluster currently comprises of 3 control planes (control nodes) and two worker nodes, each equipped with a single H100 GPU.

• Procurement of two additional GPU cards (RTX 6000 Pro) is under way to double our serving capacity for LLMs.

Hosting your own container

TODO: what people have to give us so we can host their application on Kubernetes