Uploaded image for project: 'TrueNAS'
  1. TrueNAS
  2. NAS-105845

Clean distutils dir cache before copy_tree



    • Type: Bug
    • Status: Done (View Workflow)
    • Priority: Low
    • Resolution: Complete
    • Affects Version/s: None
    • Fix Version/s: 12.0-ALPHA1, 11.3-U3
    • Component/s: None
    • Labels:


      PR: https://github.com/iocage/iocage/pull/1165

      While uninstalling a community plugin and reinstalling it with the same name there was a bug where the `overlay` folder didn't get applied to the fresh installed jail. The cause of this is probably a bug in the disutils library and how it caches the created folders.

      During debugging and testing the following exception was raised: https://github.com/python/cpython/blob/master/Lib/distutils/dir_util.py#L73 but as it is ignored and not logged during the `copy_tree` call in `ioc_plugin.py` it was very hard to find what really happen (this is why I changed the try/except to log an exception and only copy it if the overlay folder exists).

      The distutils exception was raised when it cached an overlay folder, shutil removed it during plugin removal and then while creating it again distutils thought it was already created and just skipped the creation (but the folder was not actually created)

      What could be done as an alternative is shutil lib `copytree` function inside a helper function (with extended support for ignoring existing folders), but this would introduce too many changes so I kept it simple right now.

      Note that this bug was only seen while using the UI. Please correct med if Im wrong, the UI is communicating with the jails using the `middelwared` service through a websocket channel. `middelwared` then uses `iocage` lib but from what I have seen it seems to somehow cache the python scripts as well (was forced to restart the `middelwared` service in order for my debug changes in `iocage` to take effect)? Using maybe the `.pyc` files? While running `iocage` CLI and the `destroy`/`fetch` commands I couldn't reproduce the issue, which maybe is logical, the disutils global cache variable is cleared during 2 different executions.

      Related issues:

      Should there maybe be a jira ticket for this issue as well?
      (I can squash my commits before merge but wanted to keep them during review for easy revert)

      Make sure to follow and check these boxes before submitting a PR! Thank you.






                releng Triage Team
                bugclerk Bug Clerk
                0 Vote for this issue
                1 Start watching this issue