[44net] Portal rollout lessons learned

Hello all, A few weeks ago, someone requested that ARDC share our lessons learned from the recent Portal launch. In the spirit of transparency, please find those notes below. They comprise a cleaned-up summary from an internal retrospective that occurred just over a month ago. Our retrospectives follow the following format: * What worked well? * What didn’t work as well? * What can we do better next time? (These become action items / updates to internal standard operating procedures) The “lessons learned” notes are summarized in a similar format below. As you can see, there’s a lot of room to improve, but the good news is that we are learning a lot and actively working to improve it. Feedback and questions welcome; as always, please be kind. 73, Rosy // What worked well? * Our pre-testing identified numerous bugs and UI problems, and we fixed them pre-deployment * We fixed bugs quickly, pre- and post- deployment * We provided user-facing how-to documentation * UI of new portal was vastly improved from old portal, including the implementation of a new ticketing system * ARDC internal team worked well together, ensuring team cohesion even during the most challenging portions of deployment What didn't work as well? * The code was not developed in an open-sourced fashion, eliminating opportunities for feedback and improvement pre-launch * We were not sufficiently prepared for the initial call sign verification volume (we expected 30-50 in the first 24 hours; we received over 1000 in that time frame), and thus were not able to handle it * Our DNS/Subdomain and Level of Trust policies were incomplete prior to launch, resulting in confusion with our staff and volunteers, as well as 44Net members * The initial / pre-launch communications to the mailing list did not fully prepare users for the shift in procedure. As an example, we did not specify major changes in workflow or policy from the user’s perspective. What can we do better next time? * Engage all stakeholders (in this case, regional coordinators and general 44Net users) in the design process * Provide any new or updated policies well in advance of launch, providing opportunities for feedback and discussion. * Develop code in an open-source fashion, or at least share v1 code well in advance of production-level deployment. * Ensure that we have trained multiple administrators, so that we can handle a potential overload of usage. * In our communications, make sure that we are clear about what’s changing and what’s staying the same, and speak to it from the user's perspective. Also provide notice farther in advance (e.g. 2-3 months, with reminders, rather than just 2-4 weeks) -- Rosy Schechter - KJ7RYV Executive Director Amateur Radio Digital Communications (ARDC) ardc.net