Clarifai 10.5: Gear Up Your AI: Fine-Tuning LLMs

This blog post focuses on new features and improvements. For a comprehensive list, including bug fixes, please see the release notes.
Fine-Tuning Large Language Models using the Clarifai Platform
Fine-tuning allows you to adapt foundational text-to-text models to specific tasks or domains, making them more suitable for particular applications. By training on task-specific data, you can improve model performance on those tasks.Fine-tuning lets you take advantage of transfer learning and utilize the knowledge gained from a pre-trained text model to facilitate the learning process of a new model for a related problem.Select a pre-configured model template you want to use to train on your data. You can choose from the following templates:

Llama2 7/13B or Mistral models with GPTQ-Lora, featuring enhanced support for quantized/mixed-precision training techniques.
GPT-Neo model, either the 125 million parameters version or the 2.7 billion parameters version.

Optionally, you may configure the training and inference settings to enhance the performance of your model. Checkout the complete guide on Fine-tuning Large Language Models here.

Published Coding Template, which helps to streamline the development process by facilitating efficient code completion, bug detection, refactoring, and more. App templates are pre-built, ready-to-use apps that streamline the app creation process, making it faster and more efficient. 
The Coding App template explores various coding scenarios and includes pre-built workflows tailored to address different coding use cases, employing diverse models specialized for each unique situation.

Integrated LiteLLM with Clarifai

LiteLLM is an open-source Python library that provides a unified interface to call 100+ Large Language Models (LLMs) using the same Input/Output format. It allows you to seamlessly interact with LLM APIs using the standardized OpenAI format. This integration aims to provide users with more powerful, efficient, and versatile tools for their Natural Language Processing (NLP) tasks. Check out the usage here.

Made improvements to deep fine-tuned templates
We made significant improvements to various deep fine-tuned templates, enhancing the capabilities available for training your models. The updates include:

MMClassification visual classification template: Updated from version 1.5.0 to 2.1.0, offering improved features and performance for visual classification tasks.
MMDetection visual detection template: Updated from version 2.24.1 to 3.3.0, providing advanced capabilities and optimizations for visual detection tasks.
YOLOX support added: We introduced support for YOLOX, a state-of-the-art object detection training template, further expanding the tools available for high-performance object detection.

Published new models

Wrapped Snowflake Arctic-Instruct model, a cost-effective, enterprise-focused Large Language Model (LLM) that excels in SQL, coding, and instruction-following tasks, offering open access to its advanced AI capabilities.

Wrapped GPT-4o, a multimodal AI model that excels in processing and generating text, audio, and images, offering rapid response times and improved performance across languages and tasks, while incorporating advanced safety features.

Wrapped Gemini 1.5 Flash, a cost-efficient, high-speed foundation LLM optimized for multimodal tasks, ideal for applications requiring rapid processing and scalability.

Clarifai-hosted Qwen-VL-Chat, a high-performing Large Vision Language Model (LVLM) by Alibaba Cloud for text-image dialogue tasks, excelling in zero-shot captioning, VQA, and referring expression comprehension while supporting multilingual dialogue.
Clarifai-hosted CogVLM-Chat, a state-of-the-art visual language model that excels in generating context-aware, conversational responses by integrating advanced visual and textual understanding.
Wrapped WizardLM-2 8x22B, a state-of-the-art open-source LLM, excelling in complex tasks like chat, reasoning, and multilingual understanding, competing closely with leading proprietary models.
Wrapped Qwen1.5-110B-Chat LLM, with over 100 billion parameters, demonstrates competitive performance in base language tasks, shows significant improvements in chatbot evaluations, and boasts of multilinguality.
Wrapped Mixtral-8x22B-Instruct, the latest and largest mixture of expert LLM from Mistral AI with state-of-the-art machine learning model using a mixture 8 of experts (MoE) 22b models.
Wrapped DeepSeek-V2-Chat, a high-performing, cost-effective 236 billion MoE LLM, excelling in diverse tasks such as chat, code generation, and math reasoning.
Clarifai-hosted MiniCPM-Llama3-V 2.5, a high-performance, efficient 8B parameter multimodal model excelling in OCR, multilingual support, and multimodal tasks.
Wrapped Codestral-22B-v0.1, an advanced generative LLM designed for versatile and efficient code generation across 80+ programming languages.
Clarifai-hosted Phi-3-Vision-128K-Instruct, a high-performance, cost-effective multimodal model for advanced text and image understanding tasks.
Clarifai-hosted Openchat-3.6-8b model, a high-performance, open-source LLM fine-tuned from Llama3-8b with C-RLFT, delivering ChatGPT-level performance across various tasks.

Added TypeScript (Node.js) SDK code snippets to the Use Model / Workflow modal window
If you want to use a model or a workflow for making API calls, you need to click the Use Model / Workflow button at the upper right corner of the individual page of a model or workflow. The modal that pops up has snippets in various programming languages, which you can copy and use.

We introduced TypeScript SDK code snippets as one of the tabs. Users can now conveniently access and copy the code snippets directly from the modal.

Added a feature that redirects users to their previous page after logging in

If you are logged out, you will now be taken back to the page you were on after logging in.

Introduced the ability to add users who are not organization members to teams

This enhancement allows for greater flexibility in team composition and collaboration.

Redesigned the user activation form for new Clarifai accounts

After signing up, users receive a verification email. Clicking the link in the email redirects them to the Clarifai platform to complete their profile details.

Apps / Templates
Improved the handling of unauthenticated users
If a user is not logged in or lacks access to the app:

We hide the Inputs block and its values. So, the user cannot see the inputs details on the App Overview page.
We direct the user to available public resources (e.g. models) without prompting for a login.

Added a “Reindex” checkbox in the “Change Base Workflow” section

Made improvements in the app settings page

Removed extra spaces in the tables within the API Keys and Collaborators sections on the app settings page for a cleaner and more streamlined layout.

Expanded the list of available workflows when creating a new application

You can now choose from a broader selection of base workflows to better suit your needs.

Improved ability to select a version of a resource

We introduced a standardized version selection feature across the platform, now available on both the Model-Viewer and Dataset-Viewer pages. This feature allows you to select and view specific versions of your resources using truncated digits and dates for easy identification.

Fixed an issue with the left sidebar

We fixed an issue where the left sidebar on the Input-Manager and Input-Viewer was not scrollable. Previously, this made it difficult to access all sidebar content. With this improvement, the left sidebar is now scrollable, allowing you to easily bring all parts into view.