Huggingface

​

Prerequisites

• Sign up for a Hugging Face account
• Create a Hugging Face API token

​

Setup

1. Add the Hugging Face API token to your environment variables

HUGGINGFACE_API_KEY=your-api-key

2. The Starter-Kit already includes the necessary configuration for Hugging Face in src/lib/ai/huggingface.ts, if you have selected the Hugging Face Provider via the CLI. This file initializes the Hugging Face client with your API token.

​

Usage

To use Hugging Face AI in your application, you can create an API route that interacts with the Hugging Face client. This route can be used to generate images, text, and more using the AI models provided by Hugging Face. Create a new file in your API routes directory (e.g., app/api/huggingface/route.ts). You can use the Hugging Face client to interact with the AI models and generate the desired output. Here’s an example of how to set up a route that uses the Llama 3.2 1B model to generate text:

import { kv } from '@vercel/kv';
import { huggingface } from '@/lib/ai/huggingface';
import { Ratelimit } from '@upstash/ratelimit';
import { NextRequest, NextResponse } from 'next/server';

// Allow streaming responses up to 30 seconds
export const maxDuration = 30;

// Create Rate limit
const ratelimit = new Ratelimit({
    redis: kv,
    limiter: Ratelimit.fixedWindow(5, '30s'),
});

export async function POST(req: NextRequest) {
    // call ratelimit with request ip
    const { success, remaining } = await ratelimit.limit(req.ip ?? 'ip');

    // block the request if unsuccessfull
    if (!success) return new Response('Rate limit exceeded', { status: 429 });
    const { prompt } = await req.json();

    const output = hf.textGeneration({
        model: 'meta-llama/Llama-3.2-1B',
        inputs: prompt,
        parameters: {
            max_new_tokens: 50,
            temperature: 0.7,
        },
    });
    return NextResponse.json(output);
}

This route uses the textGeneration method from the Hugging Face client to generate text using the Llama model. You can replace the model and inputs with your desired model and prompt.

​

Model Selection

Hugging Face offers a wide range of AI models that you can use in your application. You can explore the available models on the Hugging Face website.

See the Hugging Face documentation for more information on the available models and their usage.