University Gateway for Large Language Models
- We employ LiteLLM as the universal gateway to all our applications requiring access to LLMs.
- The gateway also allows you to use the provided LLMs directly with your local applications (e.g. Claude Science, OpenCode, Codex, etc.). You can obtain an API key for our centrally administered LiteLLM service using the procedure below.
- The scheme for obtaining your private API key is shown below.
The Gateway currently provides access to the following LLMs and services:
| LLM Type | Provider | Access |
|---|---|---|
| GPT-OSS-120B | CUNI (Troja) | Anyone |
| OpenEuroLLM-Czech | CUNI (Troja) | Anyone |
| Large open-source models | eINFRA* | N/A (coming soon) |
| Commercial, including Claude, etc. | OCRE | Faculty subscription needed |
(*) The models available on eINFRA run in the dedicated sensitive part of the infrastructure when accessed via the Gateway (API key).
Obtaining a private LiteLLM API key
You can connect any of your applications using the university's LiteLLM service. To do that you'll need an API key for the LiteLLM gateway.
- Connect to the OpenWebUI service using your CAS. Access to this service will be approved by the service admin.
- You'll receive an email notification informing you of the approval, including the possibility to request a LiteLLM API key.
- Login to OpenWebUI again. This registers your details in the LiteLLM database.
- Request the API key from the LiteLLM admin who will also communicate with you about your assignment to the relevant user group.
Access to commercial models may be available to you if your faculty has an active OCRE subscription, see below. Whether you personally have access to the faculty's subscription is a matter of internal faculty politics. On your request the LiteLLM admin assigns you to the desired user group. Token spendings are always traceable down to the individual users.
Token limits for the local models (CUNI)
Currently, no token usage limits are implemented but we expect that such limits will need to be implemented in the future with the growing demand for the service.
Faculty subscriptions to the OCRE service
The Open Clouds for Research Environments (OCRE) is an EU framework providing access to commercial cloud providers for research institutions. It provides access to GDPR-compliant services such as the Amazon Bedrock.
Currently, the following faculties are known to have or are in the process of obtaining OCRE subscription:
| Faculty/Unit name | OCRE subscription status |
|---|---|
| 2nd Faculty of Medicine | pending |
| Faculty of Mathematics and Physics | pending |
| Central Library | pending |
How to get an OCRE subscription
The University's Information and Communication Technologies department (UVT) can open the faculty's OCRE subscription which is available through a framework agreement between UVT and CESNET. To use that, your faculty (dean) must sign a formal specification of the services provided by CESNET. This will then allow recharging of the spendings between UVT and the faculty. This process is typically initiated by the faculty AI coordinator in collaboration with UVT.
When the agreement between your faculty and CESNET has been finalized the AI coordinator submits a request on behalf of the faculty to the UVT ServiceDesk. The request must include the following information:
- Name of the subscription, including the faculty prefix, e.g. "LF2-".
- Name of the account coordinator and their CAS ID.
- Initial financial cap in EUR for spending notifications.
The faculty's account will be setup and the spendings subsequently recharged to the faculty.
Finally, the AI coordinator contacts the LiteLLM admin to setup the faculty's budget in the system and to specify the user groups from the faculty which have access to this subscription.
Schema
The figure below shows a graphical schema of the workflows required to obtain a private LiteLLM API key and to setup a faculty OCRE subscription.
