mirror of
https://github.com/huggingface/text-generation-inference.git
synced 2025-05-28 01:32:07 +00:00
e415b690a6
* Making prefix/flashinfer the default and testing the full release tests. * Include flashinfer in the docker. * Using prebuilt. * Allowing window_left_size (dummy version). * Disabling flashinfer/prefix caching on odd head_dim * Disable prefix caching for lora. * More specific codes. * Update lock * Updating integration tests with new values with FI/FD. Remove paged as a default too, and using FD everywhere. * Update cargo lock ? * Upgrade to 1.80 because of bitstream... * Everywhere 1.80 * Forgot last default place. * Apply suggestions from code review Co-authored-by: drbh <david.richard.holtz@gmail.com> * Updated flake lock * Tmp * Upgrade resolution system for less errors in resolution. * Remove lambda for cleaner function. * Handling debugger. * OVerride the env in server tests. * Is this enough to make it work ? * This seems to be working. * Downgrade some logs. * Fixing the default for vlm. * Don't enable prefix caching on VLM just yet. * Change `add_special_tokens` in order to have the correct tokens for chat input and not (since it's super important with the prefixing now) * Fixing prefix caching for flashdecoding. * Update all models. * Fixed flashinfer version. * add_special_tokens is internal only * Fixing seqlen with the new vlms. * Fixing the issue with `add_special_tokens` not being passed around. * Fixing the test. * Removing encoder_decoder (seq2seq). * Update the chat test. * Fixing the batching tokenization in flash causal lm. * Truncating left for radix purposes. * Oops this doesn't belong here. * Put back default pure shell. * Update server tests - Default to throughput test in k6 - Use TGI_WIGGLE_ROOM to adjust wiggle room * Only n_heads / process_group.size() are necessary. * Revert the integrationt tests change (seem linked to head_size modification). * Adding error message when assert is violated. * Fixing the free algorithm to handle times where the common prefix is smaller. * Apply suggestions from code review Co-authored-by: OlivierDehaene <olivier@huggingface.co> * Update server/text_generation_server/layers/attention/common.py Co-authored-by: OlivierDehaene <olivier@huggingface.co> * Fix disabling prefix caching - Fix windowing checks. * Revert the Cohere tokenizer change (for now using a revision instead). * Fmt. --------- Co-authored-by: drbh <david.richard.holtz@gmail.com> Co-authored-by: OlivierDehaene <olivier@huggingface.co>
88 lines
2.9 KiB
YAML
88 lines
2.9 KiB
YAML
name: Server Tests
|
|
|
|
on:
|
|
pull_request:
|
|
paths:
|
|
- ".github/workflows/tests.yaml"
|
|
- "server/**"
|
|
- "proto/**"
|
|
- "router/**"
|
|
- "launcher/**"
|
|
- "Cargo.lock"
|
|
- "rust-toolchain.toml"
|
|
|
|
concurrency:
|
|
group: ${{ github.workflow }}-${{ github.head_ref || github.run_id }}
|
|
cancel-in-progress: true
|
|
|
|
jobs:
|
|
run_tests:
|
|
runs-on: ubuntu-latest
|
|
|
|
env:
|
|
SCCACHE_GHA_ENABLED: "on"
|
|
RUSTC_WRAPPER: /usr/local/bin/sccache
|
|
SCCACHE: 0.3.3
|
|
|
|
steps:
|
|
- uses: actions/checkout@v2
|
|
- name: Set up Python
|
|
uses: actions/setup-python@v1
|
|
with:
|
|
python-version: 3.9
|
|
- name: Install Rust
|
|
uses: actions-rs/toolchain@v1
|
|
with:
|
|
# Released on: 02 May, 2024
|
|
# https://releases.rs/docs/1.78.0/
|
|
toolchain: 1.80.0
|
|
override: true
|
|
components: rustfmt, clippy
|
|
- name: Install Protoc
|
|
uses: arduino/setup-protoc@v1
|
|
- name: Clean unused files
|
|
run: |
|
|
sudo rm -rf /usr/local/lib/android # will release about 10 GB if you don't need Android
|
|
sudo rm -rf /usr/share/dotnet # will release about 20GB if you don't need .NET
|
|
- name: Install sccache
|
|
run: |
|
|
curl -fsSL https://github.com/mozilla/sccache/releases/download/v$SCCACHE/sccache-v$SCCACHE-x86_64-unknown-linux-musl.tar.gz | tar -xzv --strip-components=1 -C /usr/local/bin sccache-v$SCCACHE-x86_64-unknown-linux-musl/sccache
|
|
chmod +x /usr/local/bin/sccache
|
|
- name: configure sccache
|
|
uses: actions/github-script@v6
|
|
with:
|
|
script: |
|
|
core.exportVariable('ACTIONS_CACHE_URL', process.env.ACTIONS_CACHE_URL || '');
|
|
core.exportVariable('ACTIONS_RUNTIME_TOKEN', process.env.ACTIONS_RUNTIME_TOKEN || '');
|
|
core.exportVariable('SCCACHE_GHA_CACHE_TO', 'sccache-${{runner.os}}-${{github.ref_name}}');
|
|
core.exportVariable('SCCACHE_GHA_CACHE_FROM', 'sccache-${{runner.os}}-main,sccache-${{runner.os}}-');
|
|
- name: cargo registry cache
|
|
uses: actions/cache@v3
|
|
with:
|
|
key: cargo-${{ runner.os }}-${{ hashFiles('**/Cargo.toml') }}-${{ github.sha }}
|
|
restore-keys: |
|
|
cargo-${{ runner.os }}-${{ hashFiles('**/Cargo.toml') }}-
|
|
cargo-${{ runner.os }}-
|
|
path: |
|
|
~/.cargo/registry
|
|
~/.cargo/git
|
|
- name: Install
|
|
run: |
|
|
make install-cpu
|
|
- name: Run server tests
|
|
run: |
|
|
pip install pytest
|
|
export HF_TOKEN=${{ secrets.HF_TOKEN }}
|
|
pytest -s -vv server/tests
|
|
- name: Pre-commit checks
|
|
run: |
|
|
pip install pre-commit
|
|
pre-commit install
|
|
pre-commit run --all-files
|
|
- name: Run Rust tests
|
|
run: |
|
|
cargo test
|
|
- name: sccache stats
|
|
run: |
|
|
/usr/local/bin/sccache --show-stats
|