‹ Home
Exported

TOLOGIX - ISLG App Rebuild

Laura Mindorff, COO at IndustrialHarsh Parikh, Tech Lead at DevITMelissa Cowell, General Manager at IndustrialHiren PatelMonica Martinez, MonolabMarysia RaptisPaul MoonIrit WeinfeldPatrick Harris, Web Designer at IndustrialPiyush Kanpariya, DevITRob Wiesenberg, ContegraRadomir Mladenovic, ContegraKetan Sondarva, Technical Project Manager at DevITStephanie Ducasse, Web Developer at IndustrialAkash Buch +16 others
Posted by Morgan Maguire ·

Data migration and sync for old and new ISLG applications

Hello all,

Further to Jitesh Dhuravala, DevIT Jitesh 's comment here: Re: ISLG & ILG Project Timelines - TOLOGIX - Infrastructure LawGuide (ILG) about possible confusion on the migrating and syncing data between the old (existing) ISLG application and the new ISLG application. I'm starting this message thread to get us all aligned on the approach, and then we can discuss any outstanding issue during our scheduled team call on Thursday, December 5th.

The basic task in front of us is to migrate and upload all the existing data to the new application. After the migration is complete, all content and data will be added only through the new application. However, we will be providing users with a 5-6 month beta period that will allow users to use the old and new applications simultaneously, and receive the same results in both application. As a result, we have to setup a system that will perform daily syncs between the old and new applications (i.e., daily migrations of data from the new to old application). The new application is introducing significant changes to subscribers side, and it's important that we give users a beta period where they can chose to use the old or new application as they see fit. 

There are numerous levels of complexity to achieve this task:

  1. To populate the data on the new application, we need to map out all the data sources, which are coming from both the existing application and offline spreadsheets. This process is being done through the ISLG Meta Data Field Map (see 'Source of data for field upload') where I am attempting to specify the sources for all data migrated and uploaded to the new application.
  2. To sync data back from the new to old application, we need to map out exactly if and how changes to the data in the new application affect data in the old application. Again in ISLG Meta Data Field Map (see 'Existing ISLG field for data sync), I am attempting specify if and how data changed in the new application affects data in the old application.
  3. The items above deal with mapping the data; however, we need to ensure the technical infrastructure is setup that will allow us to perform the daily syncs between the old and new applications. 
Hopefully the above gives an overview of what we need to achieve. Please share your thoughts, questions or concerns in advance of the call. If there any issues we are overlooking with this approach, please let us know.

Thanks,

Morgan

Comments & Events

Morgan Maguire, CEO Morgan Maguire, CEO
Hello all, 

Further to message above, please provide any thoughts, questions or concerns in advance of the call. I'd like the ensure we're having an informed discussion on Friday.

Thanks,

Morgan
Notified 12 people
Jitesh Dhuravala, DevIT Jitesh Dhuravala
Hi Morgan Maguire, CEO Morgan ,

After discussed and study with team, its seems like once we do data migration to new application from existing application and spreadsheet we would got idea (deep knowledge of data process) about how to do sync process, Based on migration experience we will create function/job to perform sync process from new application to old application.

We are planning about sync process should be start after data migration task successfully completed because data migration process will provide more idea/information how to do sync as per migration process experience.


Thanks,
Jitesh
Notified 12 people
Morgan Maguire, CEO Morgan Maguire, CEO
OK. Sounds good Jitesh Dhuravala, DevIT Jitesh . Let's discuss more tomorrow, but I'm glad to hear you and the team appear to be aligned with the approach.

Morgan 
Notified 12 people
Morgan Maguire, CEO Morgan Maguire, CEO
Hi Ketan Sondarva, Technical Project Manager at DevIT Ketan and Jitesh Dhuravala, DevIT Jitesh ,

Further to our call scheduled next week concerning the ISLG data migration (immediately following the call concerning dtSearch), please ensure that you examine the issues and goals above in advance of the call.

Thanks,

Morgan 
Notified 14 people
Morgan Maguire, CEO Morgan Maguire, CEO
Hello Ketan Sondarva, Technical Project Manager at DevIT Ketan and Harsh Parikh, Tech Lead at DevIT Harsh ,

During our call tomorrow, I would like to discuss how we're going to start the process of migrating and syncing data between the old and new applications in preparation for launch in November. Could you please ensure you have an internal discussion in advance of the call so that we have an informed discussion about the following issues:
  • Develop a plan for populating all content added to existing application, UIN Master List and Article Citator Master List since 9 June 2020 (i.e., we will need to perform another migration of all the content added to ISLG since the data migration in June-July)
  • Develop a plan and start migrating Documents (including PDFs, HTMLs converted from PDF and HTMLs converted from XML) and research tool data (Subject Navigator, Article Citator, etc.). We want to start the process as soon as possible with sample set of documents and data to ensure there no compatibility issues with the new application.
  • Develop a plan for how we're going to keep documents and data synced between the old and new applications for the beta period (we will have both the old and new applications operating simultaneously for several months to give users the opportunity to gradually transition to the new application). We need to identify what data and documents can be automatically synced back to the old application (e.g., document database entries and citator data) and what data will require manual upload to the old application (e.g. XML documents and PDFs with destination codes for pinpoint referencing).
Please let me know if there are any other issues that need to be flagged for discussion.

Thanks,

Morgan
Notified 6 people
Morgan Maguire, CEO Morgan Maguire, CEO
Hi Ketan Sondarva, Technical Project Manager at DevIT Ketan , Harsh Parikh, Tech Lead at DevIT Harsh and Jitesh Dhuravala, DevIT Jitesh , 

I was thinking about another issues what we'll need to consider in addition to the above:
  • As part of the migration, will need to migrate over all user and group data from the old to new application, including all legacy usage data.
  • During the beta period when users will be accessing both the old and new applications, we will need to track usage in both applications, and if possible consolidate all usage into the tracking features available in the new application.
Please integrate this into your internal discussions, and we'll discuss more during our call on Wednesday.

Thanks,

Morgan
Notified 8 people