Enable colqwen? #28

Activity

1. feuler commented on Sep 27, 2024

   

   feuler

   on Sep 27, 2024

   More actions

   Would it be easily possible to enable using https://huggingface.co/vidore/colqwen2-v0.1 with byaldi?

   Currently RAGMultiModalModel.from_pretrained("vidore/colqwen2-v0.1") leads to

   ValueError('This pre-release version of Byaldi only supports ColPali for now. Incorrect model name specified.')

   I guess in the meanwhile you could change the file https://github.com/AnswerDotAI/byaldi/blob/main/byaldi/colpali.py
   to support it. (lines 10, 67, 81+82)
   See definitions: https://huggingface.co/vidore/colqwen2-v0.1
   Some things are different. e.g.:

   • from colpali_engine.models import ColQwen2, ColQwen2Processor
   • model = ColQwen2.from_pretrained...
   • processor = ColQwen2Processor.from_pretrained...
2. jpetrantoni commented on Sep 28, 2024

   

   jpetrantoni

   on Sep 28, 2024

   Contributor

   More actions

   @feuler Thoughts on #29 ? Probably could check for correct model, init model, and init processor a little cleaner but didnt want to change too much.

   ❤️React with ❤️1Reacted by Maximilian Bode
3. feuler commented on Sep 28, 2024

   

   feuler

   on Sep 28, 2024

   More actions

   @feuler Thoughts on #29 ? Probably could check for correct model, init model, and init processor a little cleaner but didnt want to change too much.

   Thank you very much ! Just tested it, and it seems to work fine.
4. bclavie commented on Oct 3, 2024

   

   bclavie

   on Oct 3, 2024

   Contributor

   More actions

   Great contributions by @jpetrantoni who got it to work before I even recovered! Will ship a release very soon.
5. bclavie

   closed this as completedon Oct 3, 2024