Posted by Morgan Maguire · Mar 22, 2021 at 10:49 PM

Syncing/Migrating user data between legacy and new applications

Hello all,

The following is

Harsh

's proposed plan for migrating and syncing user data between the legacy and new applications in preparation for the new ISLG launch on April 6th.

Following Steps we will take to complete sync User / Group Migration.

Pull your Legacy ISLG database from Carbon 60 Server – 24th March 2021
Sync User, Group, Client, Matter and Session Detail Data in our local environment (in our local database) – 25th March to 26th March
Dev Team will do testing for following user stories after migration. – 27th March
- 20844
- 21170
- 20823
Dev IT team will make SQL Job to migrate Old Legacy Session Data to new Rebuild application. This Job will run every midnight 11:50 PM EST time on server will fetch whole day session data of user and add in to new rebuild application. – 28th March
We Will apply above migration scripts on Carbon 60 server directly on app.islg database – 30th March
It means between the new users who registered between 30th March and Launch date, those user we need to manually create on app.islg.
Your team will test user/group migration data on app.islg to verify and confirm by 31st March.

Once the above steps will be completed and verified then before one day of launch will take following user stories (stories related to old legacy application) to remove the things from old legacy application. Morgan will confirm the date to DEVIT team. For example, we will launch on 1st April then the following stories will take on 31st March and deploy on old legacy app.

21023 - Will do before one day for go live - Morgan needs to confirm the date
20852 - Will do before one day for go live - Morgan needs to confirm the date
22556 - Will do before one day for go live - Morgan needs to confirm the date

Comments & Events

Morgan Maguire, CEO

A few comments and questions about the plan above:

Harsh , regarding step #4, will we be able to distinguish between legacy application session data and new application session data within the new application, or will this all appear as the same data within session details? I don’t currently see anything on staging.islg or app.islg that indicates that distinction. Melissa and Naomi did we ever write any requirements on this?
Regarding the timing on user stories 20852 and 22556, these should occur immediately before pulling data from the legacy application on March 31st. This needs to occur before pulling the data to ensure users do no change their login information between when the data is pulled and migrated/synced with the new application. Also, user story 20852 is missing details on disabling the edit profile function for users:

image.png 184 KB • Download

Melissa and Naomi , could you please make the appropriate updates to the user story: https://industrialagency.tpondemand.com/RestUI/Board.aspx#page=board/5596457270758793748&appConfig=eyJhY2lkIjoiNEIyRDU0NTZFNkFEOUY1MUQwQzA5QzI5QjBBNjM1OTMifQ==&searchPopup=userstory/20852
Regarding the timing on user story 21023, this should occur immediately after the new application goes live on April 6th.
Harsh regarding step #6, we discussed the possibility of migrating all new users accounts created on the legacy app after you pull the data on March 31st, could you please confirm that this will be possible, so that these accounts will get integrated into the sync process? My suggestion is that we perform that final migration/sync the day before launch. Note I approximate that between 20-30 new accounts will be created during that 2 week period.

Thanks,

Morgan

Mar 22, 2021 at 11:11 PM Notified 12 people

Naomi Joanis, UX Team Lead

Morgan

,

#4 – In the story for syncing user data between the beta and legacy version there is a line to ensure the application version is recorded for an individual session. https://industrialagency.tpondemand.com/entity/20823-synching-user-data-between-beta-and

# 22556 – This story includes criteria to remove password reset and edit profile options. I've added the above screenshot to ensure it is clear which link needs to be removed.

Mar 23, 2021 at 12:02 PM Notified 12 people

Morgan Maguire, CEO

Perfect. Thanks

Naomi

.

Morgan

Mar 23, 2021 at 2:08 PM Notified 12 people

Harsh Parikh, Tech Lead

Morgan

,

As per our plan, Tomorrow 31st March, We are going to migrate User/Group Data directly on app.islg databse on Carbon60 Server. Hence, There will be chance to we should restore or backup app.islg database simultaneously.

Please make sure you and your team will be not doing any task (add/update/delete) on app.islg database by tomorrow 30tg March
(30th March 9:00 PM to 31st March 7:00 AM Vancouver Time) .

Also, For Following 2 user stories, we will update the old legacy application pages directly on www.islg.

20852
22556

Once the migration and above 2 stories will be completed then we will let you know.

Mar 30, 2021 at 7:17 AM Notified 13 people

Morgan Maguire, CEO

Great. Thanks

Harsh

.

everyone, please take note of

Harsh

's notice about no one making any modifications to the apps between 9pm Vancouver time on March 30th and 7am Vancouver time on March 31st. During this time, please do not use dev.islg, www.islg, staging.islg or app.islg.

Thanks,

Morgan

Mar 30, 2021 at 4:21 PM Notified 13 people

Nafiseh Arghandehpour, Legal Content Manager

Noted,

Morgan

. Thanks.

Mar 30, 2021 at 4:23 PM Notified 13 people

Harsh Parikh, Tech Lead

Morgan

, Just to clarify that It is 30th March 9PM to 31st March 7AM vancouuver time.

Mar 30, 2021 at 4:27 PM Notified 13 people

Morgan Maguire, CEO

Ok. Thanks

Harsh

.

Everyone, please note I've edited my post above.

Morgan

Mar 30, 2021 at 5:40 PM Notified 13 people

Harsh Parikh, Tech Lead

Morgan

,

The User /Group Migration has been completed and for session data migration every day we make one job on app.islg database which run every day 11:55 PM server time.

This SQL JOB will fetch particular day Session data and added in to app.islg database.

The Primary check of Migration is completed and move relevant stories in UAT with commenting.

Also, The following 2 stories will directly update on www.islg and move to UAT.

20852

22556

Tomorrow, we will also need to test some other stories related to migration and then will move in UAT by tomorrow.

Please note that we did migration directly only on app.islg.

Mar 31, 2021 at 1:40 PM Notified 13 people

Morgan Maguire, CEO

Ok. Great

Harsh

. Glad to hear that all went well.

I'll take a look at the data to see if there any issues.

Liam

and

Marysia

, let's plan to have a call today to ensure you understand the changes and the new system.

Thanks,

Morgan

Mar 31, 2021 at 2:44 PM Notified 13 people

Morgan Maguire, CEO

Harsh

,

I'm going to run some more tests on this over the next few days. However, I've noticed a couple of issues that need to be resolved:

All groups/users migrated from the legacy application should have "Can access both versions of the applicable" enabled. Currently, this is disabled:

image.png 130 KB • Download
Data within "Comments" field for IP address ranges within the legacy application were not migrated to the "Location(s)" field within the new application. Note that this was included in the instructions for User - IP ranges with the ISLG Data Migration Map: https://docs.google.com/spreadsheets/d/1Ho_yBR8gfCIqivmvkPy_hISkDGozdWsXxn46ImeQ-V4/edit#gid=930707860

image.png 316 KB • Download

image.png 229 KB • Download
Session data between the legacy and new applications does not line up. For example, for the subscriber "Debevoise & Plimpton" with a date range filter between 1-Mar-2020 and 1-Mar-2021 the usage on the legacy app indicates: Users: 2; Sessions: 1853; Duration: 1311:02:12; # of Pages: 17824

image.png 290 KB • Download

Whereas on the new application the same date range indicates: Users: 3; Sessions: 1853; Duration: 1311:02:12; # of Pages: 16027

image.png 177 KB • Download

What explains the difference in the number of Users and # of Pages?
The user data within the session tables for groups and users is not accurate:

Problem with data in group table (31-Mar-21).mp4 2.11 MB • Download

Thanks,

Morgan

Mar 31, 2021 at 8:59 PM Notified 13 people

Harsh Parikh, Tech Lead

Morgan

,

Following is our comment on above 4 issue.

1) All groups/users migrated from the legacy application should have "Can access both versions of the applicable" enabled. Currently, this is disabled:

- It is resolved on app.islg

3) Session data between the legacy and new applications does not line up. For example, for the subscriber "Debevoise & Plimpton" with a date range filter between 1-Mar-2020 and 1-Mar-2021 the usage on the legacy app indicates: Users: 2; Sessions: 1853; Duration: 1311:02:12; # of Pages: 17824

- It is not bug, In old legacy application the # of page count calculation is wrong. As per following screenshot you can see that # of page count is 6 but actual # of page count should be 4. In app.islg the count logic is OK.

- For Users, In legacy application the Auto-login is subscriber wise so you can see 2 Users but in our new application if there is auto-login then will display Anonymous user. Hence, the count of user is displaying 3.

image_2021_04_01T09_50_42_825Z.png 237 KB • Download

2) Data within "Comments" field for IP address ranges within the legacy application were not migrated to the "Location(s)" field within the new application. Note that this was included in the instructions for User - IP ranges with the ISLG Data Migration Map: https://docs.google.com/spreadsheets/d/1Ho_yBR8gfCIqivmvkPy_hISkDGozdWsXxn46ImeQ-V4/edit#gid=930707860

- It is not possible to migrate comment data as we are fetching City data in new application group ip ranges. In legacy application there is a comment box where admin user manually added comment data which is not matched with our City data table.

4) The user data within the session tables for groups and users is not accurate:

- We are looking into it and will update you.

Apr 01, 2021 at 10:05 AM Notified 13 people

Morgan Maguire, CEO

Ok. Sounds good,

Harsh

.

Regarding issues #2, would it be possible to pull all the comment IP range and comment data into a spreadsheet (with the associated account name), which we could use to manually update the Location(s) fields within the new application?

Thanks,

Morgan

Apr 01, 2021 at 2:42 PM Notified 13 people

Harsh Parikh, Tech Lead

Morgan

,

The 4th point issue is resolved on app.islg. : The user data within the session tables for groups and users is not accurate:

Also, Here I have attached Spreadsheet of City, Country and IP-
Range data.

City-Country-IpRanges-02April2021.xlsx 193 KB • Download

Apr 02, 2021 at 9:52 AM Notified 13 people

Morgan Maguire, CEO

Harsh

,

Further to the video below, there appears to be a problem with activation emails when accounts are created by admin users or when an account is reactivated by changing the status from "Non-Active" to "Active (Subscriber)" or "Active (Trial)". In both cases, the user did not receive the activation email requesting they create their login credentials.

Thanks,

Morgan

Problem account activation process (2-Apr-21).mp4 5.68 MB • Download

Apr 02, 2021 at 6:56 PM Notified 13 people

Morgan Maguire, CEO

Following up on above and the further two videos below, this issue appears to be much broader across the application, "Expired" status does not appear within any of the user accounts that have such status within the legacy application. Also, the counts on the homepage indicating the number of trial accounts, pending trial accounts, etc. do not appear to be accurate, nor does selecting them the relevant Review buttons take the user to accounts matching that status.

Savannah

Melissa

and

Naomi

could you please confirm whether these issues passed UAT? Note that this is particularly urgent, because

Harsh

and the team are supposed to finish all development work tomorrow. Therefore, if there any bugs in the user managements systems these need to be reported by end of day today.

Thanks,

Morgan

Problem account activation process (2-Apr-21)-2.mp4 6.1 MB • Download

Problem account activation process (2-Apr-21)-3.mp4 7 MB • Download

Apr 02, 2021 at 7:13 PM Notified 13 people

Morgan Maguire, CEO

Harsh

,

Could you please comment on the issues above concerning activation emails and expired accounts. Do we have an issue here that can be resolved before launch?

Morgan

Apr 04, 2021 at 3:34 PM Notified 13 people

Naomi Joanis, UX Team Lead

Morgan

,

Looking at the above videos, there do seem to be a couple of regression issues but others are developed slightly differently than how it works in the legacy application.

Expired Users: In this version, users are considered expired when either their group or the expiration date on the individual account has been reached – not when the password has expired. Once the user or group expiry date has been met, the user is automatically considered "non-active" status. They should still appear in the dashboard in the respective expired areas, but there isn't a status for 'expired' as these users are considered "non-active". If you would like to filter for expired users, there are filters for 'Expiry Date Range' which should accomplish the same as filtering for "expired status users" like in the legacy application. User expiry flow was tested recently so the functionality should be working as expected.
Automatic Emails for Newly Active Users: Setting a user from not active to active will simply reactivate their account with their existing credentials. This does not send an email to the user, this would be manually done by the admin. There will be an email received for ISLG admins, however, for notification of status change.
Dashboard: The dashboard area was tested however over two years ago, several different versions ago. Its possible there are some regression issues happening and that the dashboard area isn't updating properly according to the user statuses/expiry dates. I can move these cards back from done so we can perform additional tests on them to make sure any bugs that need to be logged are done properly, or maybe DevIT can look into them directly.

Apr 05, 2021 at 1:21 PM Notified 13 people

Morgan Maguire, CEO

Naomi

,

Thanks for explanation above. Could you join the call at 815am Vancouver time this morning so that we can review the above?

I'll add you to the calendar invite.

Thanks,

Morgan

Apr 05, 2021 at 1:49 PM Notified 13 people

Naomi Joanis, UX Team Lead

Morgan

,

Yes no problem. Talk then

Apr 05, 2021 at 1:53 PM Notified 13 people

Naomi Joanis, UX Team Lead

Morgan

and

Harsh

,

I just created myself as a test user on app.investorstatelawguide and I received an activation email

Screen Shot 2021-04-05 at 12.22.03 PM.png 160 KB • Download

However the subject of the email might need to be edited as well as the body content. I did not create a user with group manager permissions, and the body of the email says that I did. Besides that though the automatic emails seem to be working for me.

Apr 05, 2021 at 4:24 PM Notified 13 people

Morgan Maguire, CEO

Ok. Great. Thanks

Naomi

. We'll add this to the list of items to address post-launch.

Following-up on this and above,

Liam

and

Marysia

please take note of the following:

Expired Accounts: when a group or user account expires (i.e., the date in the Expiry Date field is reached), the status of the account will not change. These are the same requirements as the legacy application.

image.png 79.8 KB • Download

As Naomi has pointed out, this does not match the requirements in the user stories, and work will be done post-launch to correct this so that accounts that expire will automatically change to "Non Active" status.
Activating non active accounts: when a user account's status is changed from "Non Active" to "Active (Subscriber)" or "Active (Trial)", it does not reset the account and send the user a new activation email. It simply activates the account and allows the user to access the account with their existing login credentials. As a results, this can no longer be used as a means for resetting an account. We will need to examine this post-launch and modify the requirements so that we have an ability to reset an account.
Automatic Password Reset: similar to the legacy application, when the automatic password reset options are enabled on a user account:

image.png 82.3 KB • Download

the user will receive an email 7 days before the expiration date that their password needs to be reset. When the expiration date is reached, however, rather than change the account to "Expired" status, the account will be changed to "Non Active" status. Also, resetting the password after the expiration date will no longer reactivate the account, and the admin user will need to manually reactive the account by changing the status from "Non Active" to "Active (Subscriber)". However, because this restores the existing credentials without updating the password, the admin user will need to manually update the Password Expiry Date to a week later so that the user will receive the email to update their password. Post-launch, we will need to change the requirements so that the user will be able to reactivate the account by updating the password, and there should be an ability for the admin user to manually resend the user the password reset notification email.
Dashboard: the data provided in the Dashboard is not working and needs further testing:

image.png 117 KB • Download

Until further testing and work is done to confirm these data are accurate and working, please do not rely on to monitor the status of user accounts.

Please circulate any questions or concerns.

Thanks,

Morgan

Apr 05, 2021 at 4:48 PM Notified 13 people

Morgan Maguire, CEO

Harsh

,

Further to the issue concerning the automatic password reset function described above in item#3. Would it be possible for you to export a list of user within app.islg that includes data on whether the automatic password reset function is enabled on the account and the data on when the password reset is set to expire?

image.png 85.5 KB • Download

I've tried getting this data through the export use list function: https://app.investorstatelawguide.com/Admin/Subscriber/Index

image.png 92.8 KB • Download

but this didn't include data related to the automatic password reset fields:

ISLG_UserList_2021-04-06_07-13-30_CurrentUsers.xlsx 194 KB • Download

Ideally this list would include three more columns for the following:

Automatic Password Reset: Yes/No
Frequency: Every 6 months/Once per year
Password Expiry Date: Date

We are going to use this to audit these users and update the account to ensure they do not expire until we make changes to the requirements for the automatic password features in the weeks ahead.

Thanks,

Morgan

Apr 06, 2021 at 11:22 PM Notified 13 people

Morgan Maguire, CEO

Harsh

,

Further to the issues reported above concerning problems with the data presented in the group session table (see item #4: Re: Syncing/Migrating user data between legacy and new applications - TOLOGIX - ISLG App Rebuild). Further to the video below, this problem is still present in the session details that are exported from the table. Could you please ensure this is resolved as an urgent bug, because this is affecting our ability to accurately produce session data for our internal reporting purposes that we use for renewal pricing.

Thanks,

Morgan

Problem with export session details from group table (14-Apr-21).mp4 3.51 MB • Download

Apr 14, 2021 at 11:55 PM Notified 13 people

Harsh Parikh, Tech Lead

Morgan

,

The above export PDF & excel issue is resolved on both app.islg and staging.islg.

Please check and confirm.

Apr 20, 2021 at 9:40 AM Notified 13 people

Morgan Maguire, CEO

Great. Thanks

Harsh

. I'll mark this complete in the priority list.

Morgan

Apr 20, 2021 at 7:41 PM Notified 13 people

Harsh Parikh, Tech Lead

Morgan

,

Could you please mark complete this task ?

Apr 21, 2021 at 8:45 AM Notified 13 people

Morgan Maguire, CEO

Harsh

,

Yes, this was marked complete in the priority list. Note we're in a message thread, so I can't mark it complete in Basecamp.

Morgan

Apr 21, 2021 at 7:30 PM Notified 13 people