> ## Documentation Index
> Fetch the complete documentation index at: https://langchain-5e9cc07a-preview-eugene-1763577327-bfe51c2.mintlify.site/llms.txt
> Use this file to discover all available pages before exploring further.

# ChatHuggingFace

This will help you get started with `langchain_huggingface` [chat models](/oss/python/langchain/models). For detailed documentation of all `ChatHuggingFace` features and configurations head to the [API reference](https://python.langchain.com/api_reference/huggingface/chat_models/langchain_huggingface.chat_models.huggingface.ChatHuggingFace.html). For a list of models supported by Hugging Face check out [this page](https://huggingface.co/models).

## Overview

### Integration details

| Class                                                                                                                                                    | Package                                                                                    | Local | Serializable | JS support |                                                Downloads                                               |                                               Version                                               |
| :------------------------------------------------------------------------------------------------------------------------------------------------------- | :----------------------------------------------------------------------------------------- | :---: | :----------: | :--------: | :----------------------------------------------------------------------------------------------------: | :-------------------------------------------------------------------------------------------------: |
| [ChatHuggingFace](https://python.langchain.com/api_reference/huggingface/chat_models/langchain_huggingface.chat_models.huggingface.ChatHuggingFace.html) | [langchain-huggingface](https://python.langchain.com/api_reference/huggingface/index.html) |   ✅   |     beta     |      ❌     | ![PyPI - Downloads](https://img.shields.io/pypi/dm/langchain_huggingface?style=flat-square\&label=%20) | ![PyPI - Version](https://img.shields.io/pypi/v/langchain_huggingface?style=flat-square\&label=%20) |

### Model features

| [Tool calling](/oss/python/langchain/tools) | [Structured output](/oss/python/langchain/structured-output) | JSON mode | [Image input](/oss/python/langchain/messages#multimodal) | Audio input | Video input | [Token-level streaming](/oss/python/langchain/streaming/) | Native async | [Token usage](/oss/python/langchain/models#token-usage) | [Logprobs](/oss/python/langchain/models#log-probabilities) |
| :-----------------------------------------: | :----------------------------------------------------------: | :-------: | :------------------------------------------------------: | :---------: | :---------: | :-------------------------------------------------------: | :----------: | :-----------------------------------------------------: | :--------------------------------------------------------: |
|                      ✅                      |                               ✅                              |     ❌     |                             ✅                            |      ✅      |      ✅      |                             ❌                             |       ✅      |                            ✅                            |                              ❌                             |

## Setup

To access Hugging Face models you'll need to create a Hugging Face account, get an API key, and install the `langchain-huggingface` integration package.

### Credentials

Generate a [Hugging Face Access Token](https://huggingface.co/docs/hub/security-tokens) and store it as an environment variable: `HUGGINGFACEHUB_API_TOKEN`.

```python theme={null}
import getpass
import os

if not os.getenv("HUGGINGFACEHUB_API_TOKEN"):
    os.environ["HUGGINGFACEHUB_API_TOKEN"] = getpass.getpass("Enter your token: ")
```

### Installation

| Class                                                                                                                                                    | Package                                                                                    | Local | Serializable | JS support |                                                Downloads                                               |                                               Version                                               |
| :------------------------------------------------------------------------------------------------------------------------------------------------------- | :----------------------------------------------------------------------------------------- | :---: | :----------: | :--------: | :----------------------------------------------------------------------------------------------------: | :-------------------------------------------------------------------------------------------------: |
| [ChatHuggingFace](https://python.langchain.com/api_reference/huggingface/chat_models/langchain_huggingface.chat_models.huggingface.ChatHuggingFace.html) | [langchain-huggingface](https://python.langchain.com/api_reference/huggingface/index.html) |   ✅   |       ❌      |      ❌     | ![PyPI - Downloads](https://img.shields.io/pypi/dm/langchain_huggingface?style=flat-square\&label=%20) | ![PyPI - Version](https://img.shields.io/pypi/v/langchain_huggingface?style=flat-square\&label=%20) |

### Model features

| [Tool calling](/oss/python/langchain/tools) | [Structured output](/oss/python/langchain/structured-output) | JSON mode | [Image input](/oss/python/langchain/messages#multimodal) | Audio input | Video input | [Token-level streaming](/oss/python/langchain/streaming/) | Native async | [Token usage](/oss/python/langchain/models#token-usage) | [Logprobs](/oss/python/langchain/models#log-probabilities) |
| :-----------------------------------------: | :----------------------------------------------------------: | :-------: | :------------------------------------------------------: | :---------: | :---------: | :-------------------------------------------------------: | :----------: | :-----------------------------------------------------: | :--------------------------------------------------------: |
|                      ✅                      |                               ✅                              |     ❌     |                             ❌                            |      ❌      |      ❌      |                             ❌                             |       ❌      |                            ❌                            |                              ❌                             |

## Setup

To access `langchain_huggingface` models you'll need to create a `Hugging Face` account, get an API key, and install the `langchain-huggingface` integration package.

### Credentials

You'll need to have a [Hugging Face Access Token](https://huggingface.co/docs/hub/security-tokens) saved as an environment variable: `HUGGINGFACEHUB_API_TOKEN`.

```python theme={null}
import getpass
import os

os.environ["HUGGINGFACEHUB_API_TOKEN"] = getpass.getpass(
    "Enter your Hugging Face API key: "
)
```

```python theme={null}
pip install -qU  langchain-huggingface text-generation transformers google-search-results numexpr langchainhub sentencepiece jinja2 bitsandbytes accelerate
```

```output theme={null}
[notice] A new release of pip is available: 24.0 -> 24.1.2
[notice] To update, run: pip install -U pip
Note: you may need to restart the kernel to use updated packages.
```

## Instantiation

You can instantiate a `ChatHuggingFace` model in two different ways, either from a `HuggingFaceEndpoint` or from a `HuggingFacePipeline`.

### `HuggingFaceEndpoint`

```python theme={null}
from langchain_huggingface import ChatHuggingFace, HuggingFaceEndpoint

llm = HuggingFaceEndpoint(
    repo_id="deepseek-ai/DeepSeek-R1-0528",
    task="text-generation",
    max_new_tokens=512,
    do_sample=False,
    repetition_penalty=1.03,
    provider="auto",  # let Hugging Face choose the best provider for you
)

chat_model = ChatHuggingFace(llm=llm)
```

```output theme={null}
The token has not been saved to the git credentials helper. Pass `add_to_git_credential=True` in this function directly or `--add-to-git-credential` if using via `huggingface-cli` if you want to set the git credential as well.
Token is valid (permission: fineGrained).
Your token has been saved to /Users/isaachershenson/.cache/huggingface/token
Login successful
```

Now let's take advantage of [Inference Providers](https://huggingface.co/docs/inference-providers) to run the model on specific third-party providers

```python theme={null}
llm = HuggingFaceEndpoint(
    repo_id="deepseek-ai/DeepSeek-R1-0528",
    task="text-generation",
    provider="hyperbolic",  # set your provider here
    # provider="nebius",
    # provider="together",
)

chat_model = ChatHuggingFace(llm=llm)
```

### `HuggingFacePipeline`

```python theme={null}
from langchain_huggingface import ChatHuggingFace, HuggingFacePipeline

llm = HuggingFacePipeline.from_model_id(
    model_id="HuggingFaceH4/zephyr-7b-beta",
    task="text-generation",
    pipeline_kwargs=dict(
        max_new_tokens=512,
        do_sample=False,
        repetition_penalty=1.03,
    ),
)

chat_model = ChatHuggingFace(llm=llm)
```

```output theme={null}
config.json:   0%|          | 0.00/638 [00:00<?, ?B/s]
```

```output theme={null}
model.safetensors.index.json:   0%|          | 0.00/23.9k [00:00<?, ?B/s]
```

```output theme={null}
Downloading shards:   0%|          | 0/8 [00:00<?, ?it/s]
```

```output theme={null}
model-00001-of-00008.safetensors:   0%|          | 0.00/1.89G [00:00<?, ?B/s]
```

```output theme={null}
model-00002-of-00008.safetensors:   0%|          | 0.00/1.95G [00:00<?, ?B/s]
```

```output theme={null}
model-00003-of-00008.safetensors:   0%|          | 0.00/1.98G [00:00<?, ?B/s]
```

```output theme={null}
model-00004-of-00008.safetensors:   0%|          | 0.00/1.95G [00:00<?, ?B/s]
```

```output theme={null}
model-00005-of-00008.safetensors:   0%|          | 0.00/1.98G [00:00<?, ?B/s]
```

```output theme={null}
model-00006-of-00008.safetensors:   0%|          | 0.00/1.95G [00:00<?, ?B/s]
```

```output theme={null}
model-00007-of-00008.safetensors:   0%|          | 0.00/1.98G [00:00<?, ?B/s]
```

```output theme={null}
model-00008-of-00008.safetensors:   0%|          | 0.00/816M [00:00<?, ?B/s]
```

```output theme={null}
Loading checkpoint shards:   0%|          | 0/8 [00:00<?, ?it/s]
```

```output theme={null}
generation_config.json:   0%|          | 0.00/111 [00:00<?, ?B/s]
```

### Instatiating with Quantization

To run a quantized version of your model, you can specify a `bitsandbytes` quantization config as follows:

```python theme={null}
from transformers import BitsAndBytesConfig

quantization_config = BitsAndBytesConfig(
    load_in_4bit=True,
    bnb_4bit_quant_type="nf4",
    bnb_4bit_compute_dtype="float16",
    bnb_4bit_use_double_quant=True,
)
```

and pass it to the `HuggingFacePipeline` as a part of its `model_kwargs`:

```python theme={null}
llm = HuggingFacePipeline.from_model_id(
    model_id="HuggingFaceH4/zephyr-7b-beta",
    task="text-generation",
    pipeline_kwargs=dict(
        max_new_tokens=512,
        do_sample=False,
        repetition_penalty=1.03,
        return_full_text=False,
    ),
    model_kwargs={"quantization_config": quantization_config},
)

chat_model = ChatHuggingFace(llm=llm)
```

## Invocation

```python theme={null}
from langchain.messages import (
    HumanMessage,
    SystemMessage,
)

messages = [
    SystemMessage(content="You're a helpful assistant"),
    HumanMessage(
        content="What happens when an unstoppable force meets an immovable object?"
    ),
]

ai_msg = chat_model.invoke(messages)
```

```python theme={null}
print(ai_msg.content)
```

```output theme={null}
According to the popular phrase and hypothetical scenario, when an unstoppable force meets an immovable object, a paradoxical situation arises as both forces are seemingly contradictory. On one hand, an unstoppable force is an entity that cannot be stopped or prevented from moving forward, while on the other hand, an immovable object is something that cannot be moved or displaced from its position.

In this scenario, it is un
```

***

## API reference

For detailed documentation of all `ChatHuggingFace` features and configurations head to the API reference: [python.langchain.com/api\_reference/huggingface/chat\_models/langchain\_huggingface.chat\_models.huggingface.ChatHuggingFace.html](https://python.langchain.com/api_reference/huggingface/chat_models/langchain_huggingface.chat_models.huggingface.ChatHuggingFace.html)

***

<Callout icon="pen-to-square" iconType="regular">
  [Edit the source of this page on GitHub.](https://github.com/langchain-ai/docs/edit/main/src/oss/python/integrations/chat/huggingface.mdx)
</Callout>

<Tip icon="terminal" iconType="regular">
  [Connect these docs programmatically](/use-these-docs) to Claude, VSCode, and more via MCP for real-time answers.
</Tip>