Understanding packaging with conda and pip files

Hi I am a long time(ish) user of conda and its packaging world but also not super experienced in python. I am moving our team away from Conda into Pip (won’t go in to that ) but needless to say I am somewhat overwhelmed about it all.

Currently I am working on converting many (100s) of our quant packages over to Pip/pyproject.toml (we are about 7 years behind on all things python currently).

Question is: Can I include conda packages in my pyproject.toml dependencies section?

Will they be included/work with a wheel?
Do I need to declare/specify anything special?
Is noarch python and noarch generic an issue?

When I currently try run the below it says it cannot find those conda (tar.bz2) packages on pypi.org

python -m build wheel

I worry as I have little knowledge of pip packaging that we cannot use any of our conda package source code in Pip wheels anymore because its incompatible.

Really at a loss here - have googled many nights and unfortunately searching with terms like ‘pip’ and ‘conda’ brings up completely unrelated results :confused:

Maybe there is a tool I can use to convert conda projects to pip? Any help would be appreciated

1 Like

hey @pycsharpdev :wave: welcome to pyOpenSci. :pyos_animated:

Are the packages on conda that you’re trying to include as deps pure python or are there other languages?

If you have dependencies that are only on conda- you won’t be able to install them as a dependency from PyPI because as you’ve encountered, they are on a totally different packaging platform / repository - one that pip doesn’t interact with. Often conda is used because of it’s support for languages beyond Python.

if you have non python code that you want to include in a python package you’ll need to compile and bundle it with your wheel as extensions or some other build format.

this guidebook might help you in the case of a more complex build that compiles other languages into a wheel that can be published on PyPI.

the last question regarding noarch. if your package has no specific architecture build requirements then that just means that you will not need architecture-specific wheels. i may not understand the question there however.

i’ll dig a bit more to see if anyone else has more to add (without putting them on the hook to answer here over the holidays!). :slight_smile:

Can you link us to an example package so we can see which dependencies you are referring to?

2 Likes