[Matplotlib-devel] Fwd: About basemap maintenance

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
6 messages Options
Reply | Threaded
Open this post in threaded view
|

[Matplotlib-devel] Fwd: About basemap maintenance

Benjamin Root
Someone appears to have some interest in keeping up basemap maintenance. I'll leave it up to the group to decide if it is worth it or if there is a better option for Victor (cc'ed). Personally, I never attempted any sort of repackaging work like they did in conda-forge because pip installs of basemap have other difficulties (namely, the proj4 and by extension, the libgeos dependency). With conda-forge handling the packaging for me, I just never bothered even uploading new packages to PyPI.

Thoughts?

Ben Root

---------- Forwarded message ---------
From: Víctor Molina García <[hidden email]>
Date: Fri, Nov 6, 2020 at 3:47 PM
Subject: About basemap maintenance
To: <[hidden email]>


Dear Ben,

I write to you because you appear as the current (and last) maintainer of basemap. I am aware that basemap has been deprecated already quite long, but for work-related reasons I need to maintain some software that still relies on basemap.

What should I do in order to join the matplotlib team and provide myself some basemap maintenance for a bit more time? I do not plan to add new features, but I think I could help in solving the current problem of package size that prevents basemap from being available in PyPI (this is a problem for my CI stuff because I cannot rely on PyPI to install basemap as dependency and I need to provide my own basemap wheels to the deployment environment).

Essentially I workaround the package size issue in the same way as it is done in Anaconda: to split the library and separate necessary data from "optional" data. In Anaconda, you will find "basemap" (which excludes high-resolution data) and "basemap-data-hires" (with just these excluded files). I was playing a bit with the basemap setup file and I could manage to split the package in three parts:
- basemap-data: this is a package with just the basemap data of lower resolution (approx. 20 MB).
- basemap-lite: this is just the basemap source code and the compiled geoslib, and it has basemap-data as dependency (less than 1 MB).
- basemap-extras: this is the equivalent to basemap-data-hires in Anaconda (about 110 MB).

This repackaging has the advantage that at least basemap-data and basemap-lite fulfill the package size requirement of PyPI. Furthermore, since basemap-data and basemap-extras only contain data, it is possible to create for it just one universal wheel file valid for all Python versions and operating systems. Then basemap-lite needs one wheel file per Python version and operating system, but thanks to keeping the data in a separate package a lot of file redundancy is avoided.

Kind regards,

  Víctor


_______________________________________________
Matplotlib-devel mailing list
[hidden email]
https://mail.python.org/mailman/listinfo/matplotlib-devel
Reply | Threaded
Open this post in threaded view
|

Re: Fwd: About basemap maintenance

Eric Firing
Ben,

I would welcome sufficient continued maintenance to keep basemap working
smoothly with new python and mpl versions as they are released.

Eric

On 2020/11/06 12:13 PM, Benjamin Root wrote:

> Someone appears to have some interest in keeping up basemap maintenance.
> I'll leave it up to the group to decide if it is worth it or if there is
> a better option for Victor (cc'ed). Personally, I never attempted any
> sort of repackaging work like they did in conda-forge because pip
> installs of basemap have other difficulties (namely, the proj4 and by
> extension, the libgeos dependency). With conda-forge handling the
> packaging for me, I just never bothered even uploading new packages to PyPI.
>
> Thoughts?
>
> Ben Root
>
> ---------- Forwarded message ---------
> From: *Víctor Molina García* <[hidden email]
> <mailto:[hidden email]>>
> Date: Fri, Nov 6, 2020 at 3:47 PM
> Subject: About basemap maintenance
> To: <[hidden email] <mailto:[hidden email]>>
>
>
> Dear Ben,
>
> I write to you because you appear as the current (and last) maintainer
> of basemap. I am aware that basemap has been deprecated already quite
> long, but for work-related reasons I need to maintain some software that
> still relies on basemap.
>
> What should I do in order to join the matplotlib team and provide myself
> some basemap maintenance for a bit more time? I do not plan to add new
> features, but I think I could help in solving the current problem of
> package size that prevents basemap from being available in PyPI (this is
> a problem for my CI stuff because I cannot rely on PyPI to install
> basemap as dependency and I need to provide my own basemap wheels to the
> deployment environment).
>
> Essentially I workaround the package size issue in the same way as it is
> done in Anaconda: to split the library and separate necessary data from
> "optional" data. In Anaconda, you will find "basemap" (which excludes
> high-resolution data) and "basemap-data-hires" (with just these excluded
> files). I was playing a bit with the basemap setup file and I could
> manage to split the package in three parts:
> - basemap-data: this is a package with just the basemap data of lower
> resolution (approx. 20 MB).
> - basemap-lite: this is just the basemap source code and the compiled
> geoslib, and it has basemap-data as dependency (less than 1 MB).
> - basemap-extras: this is the equivalent to basemap-data-hires in
> Anaconda (about 110 MB).
>
> This repackaging has the advantage that at least basemap-data and
> basemap-lite fulfill the package size requirement of PyPI. Furthermore,
> since basemap-data and basemap-extras only contain data, it is possible
> to create for it just one universal wheel file valid for all Python
> versions and operating systems. Then basemap-lite needs one wheel file
> per Python version and operating system, but thanks to keeping the data
> in a separate package a lot of file redundancy is avoided.
>
> Kind regards,
>
>    Víctor
>
>
> _______________________________________________
> Matplotlib-devel mailing list
> [hidden email]
> https://mail.python.org/mailman/listinfo/matplotlib-devel
>

_______________________________________________
Matplotlib-devel mailing list
[hidden email]
https://mail.python.org/mailman/listinfo/matplotlib-devel
Reply | Threaded
Open this post in threaded view
|

Re: Fwd: About basemap maintenance

tcaswell
I am also supportive of Victor coming on board to help maintain basemap, but leave the ultimate decision to Ben.

I think that scheme for splitting the package up like that does make sense and leaves a path to adding a "please download just the data I need" helper.

Tom

On Fri, Nov 6, 2020 at 6:40 PM Eric Firing <[hidden email]> wrote:
Ben,

I would welcome sufficient continued maintenance to keep basemap working
smoothly with new python and mpl versions as they are released.

Eric

On 2020/11/06 12:13 PM, Benjamin Root wrote:
> Someone appears to have some interest in keeping up basemap maintenance.
> I'll leave it up to the group to decide if it is worth it or if there is
> a better option for Victor (cc'ed). Personally, I never attempted any
> sort of repackaging work like they did in conda-forge because pip
> installs of basemap have other difficulties (namely, the proj4 and by
> extension, the libgeos dependency). With conda-forge handling the
> packaging for me, I just never bothered even uploading new packages to PyPI.
>
> Thoughts?
>
> Ben Root
>
> ---------- Forwarded message ---------
> From: *Víctor Molina García* <[hidden email]
> <mailto:[hidden email]>>
> Date: Fri, Nov 6, 2020 at 3:47 PM
> Subject: About basemap maintenance
> To: <[hidden email] <mailto:[hidden email]>>
>
>
> Dear Ben,
>
> I write to you because you appear as the current (and last) maintainer
> of basemap. I am aware that basemap has been deprecated already quite
> long, but for work-related reasons I need to maintain some software that
> still relies on basemap.
>
> What should I do in order to join the matplotlib team and provide myself
> some basemap maintenance for a bit more time? I do not plan to add new
> features, but I think I could help in solving the current problem of
> package size that prevents basemap from being available in PyPI (this is
> a problem for my CI stuff because I cannot rely on PyPI to install
> basemap as dependency and I need to provide my own basemap wheels to the
> deployment environment).
>
> Essentially I workaround the package size issue in the same way as it is
> done in Anaconda: to split the library and separate necessary data from
> "optional" data. In Anaconda, you will find "basemap" (which excludes
> high-resolution data) and "basemap-data-hires" (with just these excluded
> files). I was playing a bit with the basemap setup file and I could
> manage to split the package in three parts:
> - basemap-data: this is a package with just the basemap data of lower
> resolution (approx. 20 MB).
> - basemap-lite: this is just the basemap source code and the compiled
> geoslib, and it has basemap-data as dependency (less than 1 MB).
> - basemap-extras: this is the equivalent to basemap-data-hires in
> Anaconda (about 110 MB).
>
> This repackaging has the advantage that at least basemap-data and
> basemap-lite fulfill the package size requirement of PyPI. Furthermore,
> since basemap-data and basemap-extras only contain data, it is possible
> to create for it just one universal wheel file valid for all Python
> versions and operating systems. Then basemap-lite needs one wheel file
> per Python version and operating system, but thanks to keeping the data
> in a separate package a lot of file redundancy is avoided.
>
> Kind regards,
>
>    Víctor
>
>
> _______________________________________________
> Matplotlib-devel mailing list
> [hidden email]
> https://mail.python.org/mailman/listinfo/matplotlib-devel
>

_______________________________________________
Matplotlib-devel mailing list
[hidden email]
https://mail.python.org/mailman/listinfo/matplotlib-devel


--
Thomas Caswell
[hidden email]

_______________________________________________
Matplotlib-devel mailing list
[hidden email]
https://mail.python.org/mailman/listinfo/matplotlib-devel
Reply | Threaded
Open this post in threaded view
|

Re: Fwd: About basemap maintenance

Benjamin Root
It seems like the other members were generally supportive of the idea. If you wish to pursue this, feel free to do so, I won't stop you or anyone else from doing so. My particular needs for supporting basemap ended last year, so I no longer have a compelling reason (and thus, motivation) to continue support for Basemap. Just know that once your name and email address ends up on the doc pages, you will get private requests for Basemap support approximately once a month (with no compensation). Not a major burden, but just FYI.

Ben Root


On Fri, Dec 4, 2020 at 11:47 AM Víctor Molina García <[hidden email]> wrote:
Hi Ben,

Did you think about the proposal regarding the `basemap` maintenance? In the meantime I was playing a bit on my own fork regarding this basemap split into source and data, it may look like this (my "personal" 1.3.0 version):

This does not solve the problem of GEOS as dependency, but it is one first step.

Regards,

  Víctor


El mar, 10 nov 2020 a las 15:41, Thomas Caswell (<[hidden email]>) escribió:
I am also supportive of Victor coming on board to help maintain basemap, but leave the ultimate decision to Ben.

I think that scheme for splitting the package up like that does make sense and leaves a path to adding a "please download just the data I need" helper.

Tom

On Fri, Nov 6, 2020 at 6:40 PM Eric Firing <[hidden email]> wrote:
Ben,

I would welcome sufficient continued maintenance to keep basemap working
smoothly with new python and mpl versions as they are released.

Eric

On 2020/11/06 12:13 PM, Benjamin Root wrote:
> Someone appears to have some interest in keeping up basemap maintenance.
> I'll leave it up to the group to decide if it is worth it or if there is
> a better option for Victor (cc'ed). Personally, I never attempted any
> sort of repackaging work like they did in conda-forge because pip
> installs of basemap have other difficulties (namely, the proj4 and by
> extension, the libgeos dependency). With conda-forge handling the
> packaging for me, I just never bothered even uploading new packages to PyPI.
>
> Thoughts?
>
> Ben Root
>
> ---------- Forwarded message ---------
> From: *Víctor Molina García* <[hidden email]
> <mailto:[hidden email]>>
> Date: Fri, Nov 6, 2020 at 3:47 PM
> Subject: About basemap maintenance
> To: <[hidden email] <mailto:[hidden email]>>
>
>
> Dear Ben,
>
> I write to you because you appear as the current (and last) maintainer
> of basemap. I am aware that basemap has been deprecated already quite
> long, but for work-related reasons I need to maintain some software that
> still relies on basemap.
>
> What should I do in order to join the matplotlib team and provide myself
> some basemap maintenance for a bit more time? I do not plan to add new
> features, but I think I could help in solving the current problem of
> package size that prevents basemap from being available in PyPI (this is
> a problem for my CI stuff because I cannot rely on PyPI to install
> basemap as dependency and I need to provide my own basemap wheels to the
> deployment environment).
>
> Essentially I workaround the package size issue in the same way as it is
> done in Anaconda: to split the library and separate necessary data from
> "optional" data. In Anaconda, you will find "basemap" (which excludes
> high-resolution data) and "basemap-data-hires" (with just these excluded
> files). I was playing a bit with the basemap setup file and I could
> manage to split the package in three parts:
> - basemap-data: this is a package with just the basemap data of lower
> resolution (approx. 20 MB).
> - basemap-lite: this is just the basemap source code and the compiled
> geoslib, and it has basemap-data as dependency (less than 1 MB).
> - basemap-extras: this is the equivalent to basemap-data-hires in
> Anaconda (about 110 MB).
>
> This repackaging has the advantage that at least basemap-data and
> basemap-lite fulfill the package size requirement of PyPI. Furthermore,
> since basemap-data and basemap-extras only contain data, it is possible
> to create for it just one universal wheel file valid for all Python
> versions and operating systems. Then basemap-lite needs one wheel file
> per Python version and operating system, but thanks to keeping the data
> in a separate package a lot of file redundancy is avoided.
>
> Kind regards,
>
>    Víctor
>
>
> _______________________________________________
> Matplotlib-devel mailing list
> [hidden email]
> https://mail.python.org/mailman/listinfo/matplotlib-devel
>

_______________________________________________
Matplotlib-devel mailing list
[hidden email]
https://mail.python.org/mailman/listinfo/matplotlib-devel


--
Thomas Caswell
[hidden email]

_______________________________________________
Matplotlib-devel mailing list
[hidden email]
https://mail.python.org/mailman/listinfo/matplotlib-devel
Reply | Threaded
Open this post in threaded view
|

Re: About basemap maintenance

Jody Klymak
Hi Victor,

Thanks a lot for being willing to do this.  We discussed on the developer call and this would be fine.  Thomas Caswell would need to give you commit bit on that repo.

Thanks!   Jody

On 7 Dec 2020, at 09:05, Benjamin Root <[hidden email]> wrote:

It seems like the other members were generally supportive of the idea. If you wish to pursue this, feel free to do so, I won't stop you or anyone else from doing so. My particular needs for supporting basemap ended last year, so I no longer have a compelling reason (and thus, motivation) to continue support for Basemap. Just know that once your name and email address ends up on the doc pages, you will get private requests for Basemap support approximately once a month (with no compensation). Not a major burden, but just FYI.

Ben Root


On Fri, Dec 4, 2020 at 11:47 AM Víctor Molina García <[hidden email]> wrote:
Hi Ben,

Did you think about the proposal regarding the `basemap` maintenance? In the meantime I was playing a bit on my own fork regarding this basemap split into source and data, it may look like this (my "personal" 1.3.0 version):

This does not solve the problem of GEOS as dependency, but it is one first step.

Regards,

  Víctor


El mar, 10 nov 2020 a las 15:41, Thomas Caswell (<[hidden email]>) escribió:
I am also supportive of Victor coming on board to help maintain basemap, but leave the ultimate decision to Ben.

I think that scheme for splitting the package up like that does make sense and leaves a path to adding a "please download just the data I need" helper.

Tom

On Fri, Nov 6, 2020 at 6:40 PM Eric Firing <[hidden email]> wrote:
Ben,

I would welcome sufficient continued maintenance to keep basemap working
smoothly with new python and mpl versions as they are released.

Eric

On 2020/11/06 12:13 PM, Benjamin Root wrote:
> Someone appears to have some interest in keeping up basemap maintenance.
> I'll leave it up to the group to decide if it is worth it or if there is
> a better option for Victor (cc'ed). Personally, I never attempted any
> sort of repackaging work like they did in conda-forge because pip
> installs of basemap have other difficulties (namely, the proj4 and by
> extension, the libgeos dependency). With conda-forge handling the
> packaging for me, I just never bothered even uploading new packages to PyPI.
>
> Thoughts?
>
> Ben Root
>
> ---------- Forwarded message ---------
> From: *Víctor Molina García* <[hidden email]
> <mailto:[hidden email]>>
> Date: Fri, Nov 6, 2020 at 3:47 PM
> Subject: About basemap maintenance
> To: <[hidden email] <mailto:[hidden email]>>
>
>
> Dear Ben,
>
> I write to you because you appear as the current (and last) maintainer
> of basemap. I am aware that basemap has been deprecated already quite
> long, but for work-related reasons I need to maintain some software that
> still relies on basemap.
>
> What should I do in order to join the matplotlib team and provide myself
> some basemap maintenance for a bit more time? I do not plan to add new
> features, but I think I could help in solving the current problem of
> package size that prevents basemap from being available in PyPI (this is
> a problem for my CI stuff because I cannot rely on PyPI to install
> basemap as dependency and I need to provide my own basemap wheels to the
> deployment environment).
>
> Essentially I workaround the package size issue in the same way as it is
> done in Anaconda: to split the library and separate necessary data from
> "optional" data. In Anaconda, you will find "basemap" (which excludes
> high-resolution data) and "basemap-data-hires" (with just these excluded
> files). I was playing a bit with the basemap setup file and I could
> manage to split the package in three parts:
> - basemap-data: this is a package with just the basemap data of lower
> resolution (approx. 20 MB).
> - basemap-lite: this is just the basemap source code and the compiled
> geoslib, and it has basemap-data as dependency (less than 1 MB).
> - basemap-extras: this is the equivalent to basemap-data-hires in
> Anaconda (about 110 MB).
>
> This repackaging has the advantage that at least basemap-data and
> basemap-lite fulfill the package size requirement of PyPI. Furthermore,
> since basemap-data and basemap-extras only contain data, it is possible
> to create for it just one universal wheel file valid for all Python
> versions and operating systems. Then basemap-lite needs one wheel file
> per Python version and operating system, but thanks to keeping the data
> in a separate package a lot of file redundancy is avoided.
>
> Kind regards,
>
>    Víctor
>
>
> _______________________________________________
> Matplotlib-devel mailing list
> [hidden email]
> https://mail.python.org/mailman/listinfo/matplotlib-devel
>

_______________________________________________
Matplotlib-devel mailing list
[hidden email]
https://mail.python.org/mailman/listinfo/matplotlib-devel


--
Thomas Caswell
[hidden email]
_______________________________________________
Matplotlib-devel mailing list
[hidden email]
https://mail.python.org/mailman/listinfo/matplotlib-devel



_______________________________________________
Matplotlib-devel mailing list
[hidden email]
https://mail.python.org/mailman/listinfo/matplotlib-devel
Reply | Threaded
Open this post in threaded view
|

Re: About basemap maintenance

tcaswell
Victor,

You should have an invitation to the Matplotlib org to the basemap team (just created) with write-access to the basemap repository  (I have also made Ben a "maintainer" on that team).

I do not have access to the basemap project on pypi, Ben will have to add you to that so you can publish the new packages as well.

Welcome to the team!

Tom


On Tue, Dec 8, 2020 at 4:02 PM Jody Klymak <[hidden email]> wrote:
Hi Victor,

Thanks a lot for being willing to do this.  We discussed on the developer call and this would be fine.  Thomas Caswell would need to give you commit bit on that repo.

Thanks!   Jody

On 7 Dec 2020, at 09:05, Benjamin Root <[hidden email]> wrote:

It seems like the other members were generally supportive of the idea. If you wish to pursue this, feel free to do so, I won't stop you or anyone else from doing so. My particular needs for supporting basemap ended last year, so I no longer have a compelling reason (and thus, motivation) to continue support for Basemap. Just know that once your name and email address ends up on the doc pages, you will get private requests for Basemap support approximately once a month (with no compensation). Not a major burden, but just FYI.

Ben Root


On Fri, Dec 4, 2020 at 11:47 AM Víctor Molina García <[hidden email]> wrote:
Hi Ben,

Did you think about the proposal regarding the `basemap` maintenance? In the meantime I was playing a bit on my own fork regarding this basemap split into source and data, it may look like this (my "personal" 1.3.0 version):

This does not solve the problem of GEOS as dependency, but it is one first step.

Regards,

  Víctor


El mar, 10 nov 2020 a las 15:41, Thomas Caswell (<[hidden email]>) escribió:
I am also supportive of Victor coming on board to help maintain basemap, but leave the ultimate decision to Ben.

I think that scheme for splitting the package up like that does make sense and leaves a path to adding a "please download just the data I need" helper.

Tom

On Fri, Nov 6, 2020 at 6:40 PM Eric Firing <[hidden email]> wrote:
Ben,

I would welcome sufficient continued maintenance to keep basemap working
smoothly with new python and mpl versions as they are released.

Eric

On 2020/11/06 12:13 PM, Benjamin Root wrote:
> Someone appears to have some interest in keeping up basemap maintenance.
> I'll leave it up to the group to decide if it is worth it or if there is
> a better option for Victor (cc'ed). Personally, I never attempted any
> sort of repackaging work like they did in conda-forge because pip
> installs of basemap have other difficulties (namely, the proj4 and by
> extension, the libgeos dependency). With conda-forge handling the
> packaging for me, I just never bothered even uploading new packages to PyPI.
>
> Thoughts?
>
> Ben Root
>
> ---------- Forwarded message ---------
> From: *Víctor Molina García* <[hidden email]
> <mailto:[hidden email]>>
> Date: Fri, Nov 6, 2020 at 3:47 PM
> Subject: About basemap maintenance
> To: <[hidden email] <mailto:[hidden email]>>
>
>
> Dear Ben,
>
> I write to you because you appear as the current (and last) maintainer
> of basemap. I am aware that basemap has been deprecated already quite
> long, but for work-related reasons I need to maintain some software that
> still relies on basemap.
>
> What should I do in order to join the matplotlib team and provide myself
> some basemap maintenance for a bit more time? I do not plan to add new
> features, but I think I could help in solving the current problem of
> package size that prevents basemap from being available in PyPI (this is
> a problem for my CI stuff because I cannot rely on PyPI to install
> basemap as dependency and I need to provide my own basemap wheels to the
> deployment environment).
>
> Essentially I workaround the package size issue in the same way as it is
> done in Anaconda: to split the library and separate necessary data from
> "optional" data. In Anaconda, you will find "basemap" (which excludes
> high-resolution data) and "basemap-data-hires" (with just these excluded
> files). I was playing a bit with the basemap setup file and I could
> manage to split the package in three parts:
> - basemap-data: this is a package with just the basemap data of lower
> resolution (approx. 20 MB).
> - basemap-lite: this is just the basemap source code and the compiled
> geoslib, and it has basemap-data as dependency (less than 1 MB).
> - basemap-extras: this is the equivalent to basemap-data-hires in
> Anaconda (about 110 MB).
>
> This repackaging has the advantage that at least basemap-data and
> basemap-lite fulfill the package size requirement of PyPI. Furthermore,
> since basemap-data and basemap-extras only contain data, it is possible
> to create for it just one universal wheel file valid for all Python
> versions and operating systems. Then basemap-lite needs one wheel file
> per Python version and operating system, but thanks to keeping the data
> in a separate package a lot of file redundancy is avoided.
>
> Kind regards,
>
>    Víctor
>
>
> _______________________________________________
> Matplotlib-devel mailing list
> [hidden email]
> https://mail.python.org/mailman/listinfo/matplotlib-devel
>

_______________________________________________
Matplotlib-devel mailing list
[hidden email]
https://mail.python.org/mailman/listinfo/matplotlib-devel


--
Thomas Caswell
[hidden email]
_______________________________________________
Matplotlib-devel mailing list
[hidden email]
https://mail.python.org/mailman/listinfo/matplotlib-devel


_______________________________________________
Matplotlib-devel mailing list
[hidden email]
https://mail.python.org/mailman/listinfo/matplotlib-devel


--
Thomas Caswell
[hidden email]

_______________________________________________
Matplotlib-devel mailing list
[hidden email]
https://mail.python.org/mailman/listinfo/matplotlib-devel