• 0 Posts
  • 3 Comments
Joined 1 year ago
cake
Cake day: September 16th, 2023

help-circle

  • tequinhu@lemmy.worldtoScience Memes@mander.xyzPandas
    link
    fedilink
    English
    arrow-up
    3
    ·
    edit-2
    3 months ago

    I didn’t know about lazy loading, that’s cool!

    Then I guess that the meme doesn’t apply anymore. Though I will state that (from my anedoctal experience) people that can use Panda’s most advanced features* are also comfortable with other data processing frameworks (usually more suitable to large datasets**)

    *Anything beyond the standard groupby - apply can be considered advanced, from the placrs I’ve been

    **I feel the urge to note that 60Mb isn’ lt a large dataset by any means, but I believe that’s beyond the point


  • tequinhu@lemmy.worldtoScience Memes@mander.xyzPandas
    link
    fedilink
    English
    arrow-up
    10
    ·
    3 months ago

    It really depends on the machine that is running the code. Pandas will always have the entire thing loaded in memory, and while 600Mb is not a concern for our modern laptops running a single analysis at a time, it can get really messy if the person is not thinking about hardware limitations