Tuesday, June 27, 2017

Change Log: 2.11.359

...and we're back. Deployment of the new release had the Manager down for about 10 minutes, and everything is looking really nice in the aftermath.

In terms of what's in this release, let me just say that 2.11.359 is a big boy: it took me a couple of weeks to put it together and it improves the Manager quite a bit. What follows is a quick breakdown of everything included in this major release.

Installation/deployment. This release fixes the numerous sysadmin-type problems/issues with the last one, which revised the way the manager is deployed (i.e. the way its various services and processes are initialized). Long story short, the last release made a lot of assumptions about the target system that ended up not being true: the installer now displays more flexibility/adaptability in those critical areas.

Settlement Sheet upgrade! Release 2.11.359 one also includes some important usability improvements, the most significant of which is the implementation of an angularjs-based locations picker on the Settlement Sheet, which will finally mean (drum roll, please) that the Settlement Sheet will no longer reload/refresh when locations are added/removed using the Settlement Sheet controls.

Back-end security and performance enhancements. There are some improvements on the back-end as well, including some new code in the legacy webapp and the API that will help transition the old application to the API's security model and allow us to start locking down parts of the API. 2.11.359 also migrates a number of legacy webapp methods/features to the API, which will a.) make them perform more reliably and b.) get us closer to our alpha release of The Watcher.

Fixes and corrections. As far as fixes go, this release also resolves all of the issues that were reported during my two-month hiatus, including several issues with the Location picker, the resurrection bug affecting the Controls of Death and the way that API failures happen.

At this point, the most important short-term goals, in terms of getting the API ready to work with The Watcher and modernizing the legacy webapp are these:
  • refactor all Javascript used by the legacy webapp to no longer use the modifyAsset() function, i.e. the function which makes a call to the legacy app to update a survivor or settlement, and replace all of those methods with API calls.
  • make the "private" API routes, i.e. the ones that manage user-created assets (Survivors and Settlements), require JWT authorization. 
  • deprecate all legacy webapp code that modifies user assets and replace it with an equivalent API method.
Finally, due to some outside complications, I will be taking another break from active development/support of both the Manager and the API from July 17th through August 17th.

Which means that the agenda for the next few weeks of development is to make some of the major enhancements that we need for The Watcher and stabilizing the legacy webapp so that it doesn't need any serious attention while I am out of office.

As usual, hit the jump for the full change log.

Thanks again for using the Manager!


Corrections and Fixes

  • install.sh will not work (i.e. exit automatically) if it cannot find the Gunicorn binary (using which).
  • install.sh no longer creates settings_private.cfg files as root, breaking the install
  • The API and the World daemon now write their PID files to the general application PID file dir, rather than to the local check-out; install.sh no longer creates a run directory in the local checkout.
  • Fixed an issue where the API's world.WorldDaemon functions for checking the PID directory could throw a false positive error if the run path was specified incorrectly in settings.cfg for the v2/API
  • Addressed a usability/UX issue where live survivors could be resurrected, resulting in random/cascading UI failures. -JM
  • API failures in settlementSheet.js now log as errors (rather than just console.log).
  • Fixed logging in the API settlement asset conversion methods that mistakenly reported innovation conversion failures as location conversion failures.
  • Fixed a bug affecting PotSun campaigns where Survival Actions unavailable in the campaign appeared on the Survivor Sheet.
  • Addressed an issue where Light-Forging items in Settlement Storage were barely visible due to bad color choices (this was actually a typo in the legacy game_assets.py asset definition for the location).
  • Addressed an issue where the "Blacksmith" and "Barber Surgeon" locations could sometimes not be chosen from the Settlement Sheet location picker. -jmiller
  • Fixed an issue where the Settlement Event log would record user object IDs for certain events, e.g. unsetting settlement principles.
  • The Settlement Event log now uses asset names (instead of handles) when recording Location and Innovation level updates.
  • Fixed an order-of-operations bug in the legacy webapp that caused false positive Javascript errors to show up in the console while attempting to render the Campaign notes/players controls.

Application Improvements

  • Both settings.cfg and install.sh have been updated so that the legacy webapp automatically determines the uid/gid of the Linux user running the application server.
  • world.api_killboard_to_html() now uses the validate_api_returns() method (which went out in the last release) to fail gracefully.
  • install.sh has been improved:
    • it now sets the path to the Gunicorn binary.
    • it lets the user set the install mode at run time.
    • it no longer writes a settings_private.cfg file in the API root; it copies from the /src directory as the install user instead
    • it writes an api.thewatcher.service file to the API root during the install.sh run
    • it can now update the v2 settings.cfg as well
    • PID and logging root directories are SGID now, so all files created should be accessible
  • Moved the install code/config stubs to /src dir
  • Created clone_user.py which is a simple development CLI utility that accepts a production webapp user's OID, dials the webapp, pulls their pickle down and imports it to the local app.
  • Removed/deprecated the qa_user.pickle since it's so easy to just grab/clone actual production users down to the local now.
  • admin.py now imports from a pickle in memory or on the file system (and dynamically detects what you're trying to do)
  • Survivor Sheet Controls of Death now only show the "resurrect" button if the survivor is dead.
  • Deprecated legacy webapp Endeavor Token functionality:
    • removed assets.get_endeavor_tokens() method
    • removed support for Endeavor Token operations from assets.Settlement.modify() 
    • assets.render_html_summary() no longer has anything to do with the Endeavor Token UI on the Campaign Summary
    • removed assets.update_endeavor_tokens()
    • legacy app html.py templates get ET information from the API now
  • kdmManager.js has been updated so that the app.controller controller's postJSONtoAPI method will POST using the current cookie's JWT as its "Authorization" header.
  • Changes to the settlement's total endeavor tokens are now tracked in the Settlement Event Log (finally).
  • Implemented an Angular.js location picker on the Settlement Sheet and deprecated the old, form-based controls. 
    • Updated legacy assets.Settlement.get_endeavors() to use handles for locations
    • Deprecated legacy JS function locationsController.setLocationOptions() method from settlementSheet.js.
    • Updated the Campaign Summary HTML controls to use location handles
  • Deprecated assets.Survivor.get_survival_actions() as well as assets.Settlement.get_survival_actions() and the associated legacy HTML form and banished a couple hundred lines of code to the land of Wind and Ghosts. 
  • The kdmManager.getJSONfromAPI() method has been improved to use JWT Authorization in the header of its request.
  • api.route_to_dict() now processes a Session object instead of just accepting a raw 'access_token' string and uses the Session object to refresh/set the Authorization header as necessary.
  • session.py has been updated to get (and then set) API assets using a api.route_to_dict() call that includes a Session object.

API Improvements

  • api.collection_action now supports JWT authorization! It doesn't 100% require it yet, and will (for the time-being) continue to work without a token. 
  • models.settlements.normalize_data() now creates the settlement["endeavor_tokens"] key if it is unset. 
  • Created models.settlements.update_endeavor_tokens() method to replace the functionality deprecated from the legacy app.
  • API now supports Endeavor Token operations (via posted JSON)
  • Rewote models.locations.Assets class:
    • Removed the whole 'subtype' attribute thing because that was just...idiotic and I can't even believe I thought that was a good idea to begin with.
    • Enhanced DRYness and config-driven code by replacing asset import hard-coding with iteration over the module.
    • Removed placeholder code and notes from the module, including manual logger import and my assorted musings on how to improve the module.
    • When initialized, the class just sets 'root_module' and does vanilla importing.
  • Settlement Sheet location management operations now supported by the API:
    • models.settlements.add_location() 
    • models.settlements.rm_location() 
    • models.settlements.set_location_level()
  • Improved world.WorldDaemon.set_pid() so that it checks for the existence of the PID in the PID file and, if it doesn't find it, removes the PID file and unsets the PID (i.e. because the process is dead). Should fix our zombie daemon problem.
  • Also touched up world.WorldDaemon.stop() to use os.kill() so that it sends a signal 15 to the process via python, rather than doing the system call. Removed system call code from the module.
  • The World daemon may now be started by root (who will hand it off to the daemon_user specified in the API settings.cfg file).
  • The API's init/controller script server.sh now starts and stops the world daemon automatically.
  • All API logging (including World daemon, server, etc.) takes place in the main logging root, i.e. OUTSIDE of the project code check out
  • Enhanced settings.py to accept CLI arguments; built a function for setting values from CLI
  • The World Daemon will no longer attempt (and fail) to start itself if it is already running.
  • The settings.py module now only looks in its own directory for its settings.cfg or settings_private.cfg files and will raise a big, messy OSError exception if it cannot find those files within the directory where it lives.
  • The Survivor serialize() method now returns available survival actions as a top-level element.
  • The Survivor Sheet display for Survival Actions now uses the API + an angularjs front-end. 
  • Created hooks for Abilities and Impairments in the assets.abilities_and_impairments.Assets() class. I also started porting some A&Is from the legacy webapp game_assets.py
  • Revised assets.abilities_and_impairments.Assets() to just use the vanilla asset initializer from Models.py by setting a 'root_module' on init.
  • Re-implemented Settlement Sheet locations so that Settlement locations are stored in MDB as handles now, rather than names:
    • Removed vestigial/deprecated counting code from Innovation and Location conversion methods.
    • Innovation and Location conversion methods now record failure in the Settlement Event log. This should almost never happen, but I think we want to squawk at users when it does.
    • Deprecated game_assets.locations_options element from serialized settlements, because it violates the data model (and was pure for a legacy support operation that is no longer required)
  • Revised the approach to representing Survival Actions for both the settlement and survivor JSON -ckester:
    • Renamed both models.settlements.get_available_survival_actions() and its corollary in models.survivors to get_survival_actions(), which is more concise and accurate.
    • models.settlements.get_survival_actions() now accepts a 'return_type' kwarg and can return JSON-style output, rather than dictionary style output.
    • Survival Actions are now defined by the campaign type (i.e. they're an attribute of the campaign asset definition).
    • Refactored assets.settlements.get_survival_actions() to use the campaign attribute and compare it to innovations to set what's available.
    • models.survivors.get_survival_actions() now adds survival actions that are available because of A&Is.
  • models.settlements.get_available_assets() now accepts a kwarg called 'exclude_types' that takes a list and uses it to filter/exclude arbitrary assets based on their 'type' attribute.
  • Models.AssetCollection() base class now supports a "filter" method that allows arbitrary asset collection objects to drop assets after initialization.
  • Models.AssetCollection() base class now allows modules sub-classing it to set self.root_module prior to sub-classing to automatically set assets based on the 'root_module' attrib.
  • Cleaned up the documentation of Models.AssetCollection.__init__() so that it is a.) moved out of in-line comments and b.) clearly reflected in the organization of the code itself.
  • Settlement Event Log now records who adds/removes innovations on the Settlement Sheet.
  • Added check_request_params() to the Models.UserAsset base class methods to enhance the DRYness of some of the mdoels.settlements methods.
  • The 'innovation_levels' and 'location_levels' attributes of the MDB settlement document are now mandatory and applied by models.Settlement.normalize_data().
  • Refactored Models.get_asset() to use the self.asset dictionary get() method rather than the old try/except logic. The code is a little cleaner now. I also removed the user-friendliness part about adding the handle to the asset if the asset doesn't define its own handle on the basis that it is implicit rather than explicit.




No comments:

Post a Comment