Automatically map deduplicated safetensors weights to their original values (#501)

Browse Source

# What does this PR do?

This PR automatically points tensors that were removed due to
deduplication to their still existing twin.

In `server.text_generation_server.utils.convert.py#convert_file`,
tensors that have a value equal to another tensor are removed from the
list of weights. Their name, and the name of the still-existing "twin"
are logged to the "metadata" dictionary. However, this dictionary was
not yet used during loading. This requires explicit in-code remapping
when loading the models (as mentioned in the docstring).

This PR adds some simple code to check, during loading, if a weight is
one of those removed weights. It then automatically retrieves the values
of its still-existing "twin" instead.

## What does this fix?
We currently cannot load `h2oai/h2ogpt-oig-oasst1-falcon-40b` with the
unmodified server, since the `transformer.word_embeddings.weight` weight
is equal to `lm_head.weight` and is automatically removed. The falcon
code, however, still expects this weight to exist. I could have also
added some extra checks to the model itself, though that would only be a
workaround.

## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the
other checks if that's the case).
- [ ] Did you read the [contributor
guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the
[forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes?
Here are the
[documentation
guidelines](https://github.com/huggingface/transformers/tree/main/docs),
and
[here are tips on formatting
docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?

Anyone in the community is free to review the PR once the tests have
passed. Feel free to tag
members/contributors who may be interested in your PR.

<!-- Your PR will be replied to more quickly if you can figure out the
right person to tag with @


@OlivierDehaene OR @Narsil

 -->

---------

Co-authored-by: Vincent Brouwers <vincent.brouwers@ing.com>
Co-authored-by: Nicolas Patry <patry.nicolas@protonmail.com>

...

This commit is contained in:

Vincent Brouwers 2023-08-02 19:55:03 +02:00 committed by GitHub

parent 7766fee9b1

commit 9bcac46d00

No known key found for this signature in database

GPG Key ID: 4AEE18F83AFDEB23

Show all changes Ignore whitespace when comparing lines Ignore changes in amount of whitespace Ignore changes in whitespace at EOL

Show Stats Download Patch File Download Diff File Expand all files Collapse all files

Diff Content Not Available