Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fixed memory leak #79

Merged
merged 14 commits into from
Jan 24, 2023
Merged

Fixed memory leak #79

merged 14 commits into from
Jan 24, 2023

Conversation

aburrell
Copy link
Owner

@aburrell aburrell commented Jan 23, 2023

Description

There was a memory leak due to a lack of freeing PyObject pointers in the C code. Fixes #74 and #75 by clarifying the existing logger message.

Type of change

Please delete options that are not relevant.

  • Bug fix (non-breaking change which fixes an issue)

How Has This Been Tested?

Make sure to re-install AACGMV2 before testing, since the C code has to be re-compiled. Also added two new unit tests that would have segfaulted previously.

import timeit

# Yields 15.579 s on my system
timeit.timeit("import aacgmv2; import datetime; import numpy as np; rando_lat = np.random.uniform(low=-90,high=90,size=6000); rando_long = np.random.uniform(low=-180,high=180,size=6000);aacgmv2.convert_latlon_arr(rando_lat, rando_long, 350.0, datetime.datetime(2015, 5, 5), method_code='G2A')", number=1000)

# Yields 1866.4816 on my system
timeit.timeit("import aacgmv2; import datetime; import numpy as np; rando_lat = np.random.uniform(low=-90,high=90,size=6000); rando_long = np.random.uniform(low=-180,high=180,size=6000);aacgmv2.get_aacgm_coord_arr(rando_lat, rando_long, 350.0, datetime.datetime(2015, 5, 5), method='G2A')", number=1000)

# Yields x s on my system
timeit.timeit("import aacgmv2; import datetime; import numpy as np; rando_lon = np.random.uniform(low=-180,high=180,size=6000);aacgmv2.convert_mlt(rando_lon, datetime.datetime(2015, 5, 5), m2a=False)")

Test Configuration:

  • Operating system: OS X Big Sur
  • Version number: Python 3.9
  • Any details about your local setup that are relevant

Checklist:

  • Make sure you are merging into the develop (not main) branch
  • My code follows the style guidelines of this project
  • I have performed a self-review of my own code
  • I have commented my code, particularly in hard-to-understand areas
  • I have made corresponding changes to the documentation
  • My changes generate no new warnings
  • I have added tests that prove my fix is effective or that my feature works
  • New and existing unit tests pass locally with my changes
  • Any dependent changes have been merged and published in downstream modules
  • Add a note to Changelog.rst, summarising the changes
  • Add yourself to AUTHORS.rst

Closed open PyObject variables in convert_latlon_arr and removed objects that were stolen too many times.
Freed memory in remaining array functions.
Updated the changelog.
Added unit tests for large arrays.  Previously, arrays this large caused seg faults due to memory leaks.
Added unit tests for the MLT array conversions.  Also added test for RuntimeError messages.
Made the warning message clearer that the problem is with the input, not the code.
Added an additional logging message test and improved style of existing logger tests to conform to pytest standards.
@aburrell aburrell added the bug label Jan 23, 2023
Remove the sys import, as it's no longer used.
Removed `assert` calls to appease codacy.
Improved the comments and syntax, attempting to find errors.
Updated the strict flag.
@aburrell
Copy link
Owner Author

The code occasionally segfaults (regularly in tests). Trying to figure out why.

Re-add some of the necessary metadata back to the setup.cfg file.
Don't attempt to free data input into the C wrappers, these are needed outside of the C functions.
Try to remove some of the new metadata lines as make is failing.
@coveralls
Copy link

coveralls commented Jan 24, 2023

Pull Request Test Coverage Report for Build 3998919162

  • 70 of 74 (94.59%) changed or added relevant lines in 3 files are covered.
  • No unchanged relevant lines lost coverage.
  • Overall coverage increased (+0.5%) to 92.655%

Changes Missing Coverage Covered Lines Changed/Added Lines %
aacgmv2/tests/test_py_aacgmv2.py 47 48 97.92%
aacgmv2/tests/test_c_aacgmv2.py 21 24 87.5%
Totals Coverage Status
Change from base Build 3972015180: 0.5%
Covered Lines: 1122
Relevant Lines: 1199

💛 - Coveralls

@coveralls
Copy link

Pull Request Test Coverage Report for Build 3998919162

  • 70 of 74 (94.59%) changed or added relevant lines in 3 files are covered.
  • No unchanged relevant lines lost coverage.
  • Overall coverage increased (+0.4%) to 92.58%

Changes Missing Coverage Covered Lines Changed/Added Lines %
aacgmv2/tests/test_py_aacgmv2.py 47 48 97.92%
aacgmv2/tests/test_c_aacgmv2.py 21 24 87.5%
Totals Coverage Status
Change from base Build 3972015180: 0.4%
Covered Lines: 1152
Relevant Lines: 1231

💛 - Coveralls

@aburrell
Copy link
Owner Author

@dinsmoro would you mind testing this branch?

@dinsmoro
Copy link

This branch fixes the memory leak issue! Thanks for solving it!

@aburrell aburrell merged commit e182e2e into develop Jan 24, 2023
@aburrell aburrell deleted the memory_leak branch January 24, 2023 21:48
@aburrell aburrell restored the memory_leak branch December 30, 2024 19:55
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants