✔ Problem with <linknum> tags in XML Agreements
Completed by Irit W.
- Assigned to
-
Harsh P.
Jitesh D.
Ryan K.
- Due on
- Notes
-
Further to the fix we implemented in this to-do: Problem with loading XML documents - TOLOGIX - ISLG Maintenance, we have discovered an additional issue with XML Agreements. Further to the video below, for a long period of time the XML Agreement uploader has not been converting <linknum> tags into the appropriate tags with agreementIDs and kyIDs. As a results, a large number of documents still have the generic <linknum> tags in the XML code, and thus the appropriate links to the Article Citator are not getting made on the front-end of ISLG. This problem with the uploader appears to be resolved in the recent fixes above; however, we now need to retroactively fix all the XML documents that currently have generic <linknum> tags in XML Agreements.
To further illustrate what I'm requesting, we need to ensure that any XML document in XML Agreements that currently has <linknum> tags like this: https://www.investorstatelawguide.com/CoreComponents/Agreements_BitsEdit?toc=editXml&agreementID=543
are converted into tags with agreementIDs and kyIDs like this: http://dev.investorstatelawguide.com/CoreComponents/Agreements_BitsEdit?toc=editXml&agreementID=1536
This will need to be done across the entire XML Agreement database. Please confirm whether this will be possible, and we'll implement the solution on dev.islg.
Yes, It is possible, we will access each XML document under the XML agreements and convert all <linknum> to <link> tag will agreementIDs and keyIDs. We will let you know once it will done.
Thanks,
Jitesh
Morgan
We have matched our www.islg 's database and production environment 's XML files and found that total 134 files which don't have agreementIds and keyIDs in linknum tag.
We have replaced those file linknum tag with link tag and assigned agreementIDs and KeyIds on both www.islg and dev.islg.
Please check and confirm.
Following, I have attached spread sheet which contain 134 XML file name.
Note that I'm traveling today, so I won't be able to take a look until later this evening. Also, we're doing demos of the site tomorrow so if you could do a general survey of the site, it would be appreciated.
Thanks,
Morgan
We have to perform this task on www.islg becasue we don't have all xml files on dev.islg.
We had already taken backup of all XML files so if we will face any issue then will restore the backup.
I will check the front-end side that and let you know.
We have restored old XML files on www.islg as we have found one small issue that duplicate paragraph numbers are generated.
We will resolve this issue by tomorrow on dev.islg then will migrate on ww.islg.
Thank you for restoring the old XML's on www.islg. Also, please do not make any further changes on www.islg until I give further instructions when I'm back in the office next week. I don't want there to be any changes while we perform problems while we're performing demos over the next 5 days.
In the meantime, on dev.islg, in addition to the duplicate paragraph references you noted above, there is a problem with converted <linknum> tags. Further the screenshot below, the <linknum> tags have been replaced with tags that have the same kyIDs accross the entire document: http://dev.investorstatelawguide.com/CoreComponents/Agreements_BitsEdit?toc=editXml&agreementID=260
I'm surprised this was implemented as part of your fix, but please ensure that every link tag has a unique kyID to ensure in combination with the agreement ID, the tag can be uniquely identified across the database. Not having unqiue IDs has affects on connecting the link to the appropriate entry in the Article Citators.
Thanks,
Morgan
We have found that Kyid is same across those XML file where agreementIds and KyIds are not set in <linkum> tag on production environment.
As and example, If you check the Kyid of XML file "BIT\0301" on www.islg then you can see that Kyid is set 1 in whole XML file.
We need an incremental KyIds in XML files.
Yes, incremental kyIDs are required. Could you please ensure that any of converted <linknum> tags are converted into incremental kyIDs, and I assume this will resolve the problem. Please perform the conversion on dev.islg for the documents you have already converted.
Thanks,
Morgan
Morgan
We are working on this task and developed one program which replaced all <linknum> tag with <link> and assign Agreement_Id and incremental KyIds. Currently we are checking this program with 140 XML files and also check those XML file on front-end side.
Let you know once it will be successfully completed.
Morgan
Total we have found 130 XML files which has <linknum> tag without agreementids and Kyids and which are currently using in www.islg.
We have replaced <linknum> tag with <link> tag and alos applied Agreementids and incremental kyids on attached 130 XML files on dev.islg.
Please check and let us know the feedback.
Thanks!
Ryan
I'll take a look when I get into the office later today. I'm current in the ER, but should be in this afternoon.
Thanks,
Morgan
I checked a few listed in the linknum_XML.zip and found the ones I checked have been fixed properly on dev.islg.
Thanks,
Paul
Morgan
However, Irit and I discovered a few issues that may have been related to this fix especially how non-English characters are displayed (e.g.,BIT/0811 and BIT/0314). Should this be posted separately?
Paul
Ok. Great. I'll take a second look this afternoon.
Yes, please post the non-English character problem. However, is this affecting both dev.islg and www.islg, or just dev.islg?
Morgan
It is only affecting dev.islg at the moment.
Paul
Thanks,
Morgan
Attached is the list of XMLs that have non-English character problems on the Dev site. You will see that a few are highlighted in green. These look fine on the Dev site and ISLG Site. Also attached are screen captures comparing the problem on the Dev site vs the ISLG site.
Thanks,
Irit
Further to Irit's reports above, could you please ensure these issues with the non-English characters are resolved as part of the fix. It looks like the conversion of the linknum tags had some effect on these documents.
Thanks,
Morgan
I also noticed that the duplicate links are appearing for the sub-paragraphs: https://dev.investorstatelawguide.com/Treaties/AnnotDocument?toc=annotSection&agreementID=260&tabcontent=&cat=bit
Thanks,
Morgan
The highlighted sub-paragraph does not display duplicate links. Actually the number beside Sub-Paragraph number is indicating the count of document.
For example : In Sub-Paragraph 1. [1], The 1. is sub-paragraph number and [1] is indicating the count of documents.
My apologies, I did forget to test the front-end links. Noted for next testing round. I've also confirmed Harsh's note above.
Thanks!
Ryan
Thanks,
Morgan
We have resolved non-English character issues on dev.islg which Irit has mentioned in word document.
From above mentioned red marked XML in docuemnt, The Following XML structure' s is not correct. Hence, When you are going to open following XML from Treaty & Rule Page on dev.islg then it will throw the error.
We need correct formatted XML file so we can replace with it.
Can you please provide the original XML for BIT/0208. So that
Thanks,
Morgan
Thanks,
Irit
I do not see the non-English character problem resolved on dev.islg. From Irit's list, I checked three XMLs on dev.islg and still see the issue as shown below. I highlighted some of the issues.
BIT/0065 Pakistan-Switzerland [French]
https://dev.investorstatelawguide.com/Treaties/AnnotDocument?toc=annotSection&agreementID=238&tabcontent=&cat=bit
BIT/0048 Italy-Morocco [Italian]
https://dev.investorstatelawguide.com/Treaties/AnnotDocument?toc=annotSection&agreementID=230&tabcontent=&cat=bit
BIT/0021 - Chile-Malaysia [Spanish]
https://dev.investorstatelawguide.com/Treaties/AnnotDocument?toc=annotSection&agreementID=226&tabcontent=&cat=bit
Please let us know when this problem is fixed.
Thanks,
Paul
To Resolve non-English character problem, You need to delete existing XML file and upload new XML file again through our application.
Please use attached XML files for again upload through our application.
Please follow following step :
1) Delete XML file
2) Find XML file from attached zip file
3) Upload through our application
Please try on dev.islg and let us know the feedback.
Thanks,
Paul
I still see the non-English character problem on dev.islg. I did a test with the three XML documents that Paul sent you on July 29th: BIT/0065, BIT/0048, and BIT/0021. I deleted the XML file as per your instructions. I then uploaded the XML file provided in your zip file. The non-English character problem looks a bit different now, as you will see in the screen capture below. I compared the screen capture from July 29 to the one from today for BIT/0065. You will see the "?" symbol highlighted in today's:
BIT/0065 Pakistan-Switzerland [French]
31-JULY screen capture:
BIT/0048 Italy-Morocco [Italian]
31-JULY screen capture:
BIT/0021 - Chile-Malaysia [Spanish]
31-JULY screen capture:
Thanks,
Irit
Is there any update with this item?
Please let us know.
Thanks,
Paul
Sorry for late reply, We are working on this to convert non-character English file through encoding but we are facing some problem. The work is in progress.
We will keep you updated.
Paul
Would it be possible to get an update on this to-do?
Thanks,
Morgan
We are working on this. If we resolve this issue then it is affecting other documents so we are finding general solution.
Will try to complete before end of this week.
Morgan
Thanks,
Morgan
We have resolved the non-English character problem on dev.islg. Please check and let us know the feedback.
Could you test the dev.islg and let us know whether this issue is resolved or not?
Thanks,
Paul
I spotted issues. I will take a screen shot and respond in basecamp.
https://dev.investorstatelawguide.com/Treaties/AnnotDocument?toc=annotSection&agreementID=1818&tabcontent=&cat=bit
21-AUGUST screen capture after deleting/ re-uploading the XML:
BIT/0811 Belgium - Luxembourg - Poland BIT (1987) [French]
Link to dev site:
21-AUGUST screen capture before deleting the XML:
21-AUGUST screen capture after deleting/ re-uploading the XML:
BIT/0133 Argentina - France BIT (1991) [French]
21-AUGUST screen capture after deleting/ re-uploading the XML:
Irit
We are looking into it. Currently, We are facing problem like if we resolve the non-English characters then it creates problem in linknum tags and if we resolve the linknum tags then it creates problem then in non-English characters.
We are trying to resolve it and will update you soon.
We have resolved above 3 files characters issue on dev.islg. Please check and confirm.
Now, Here, I have attached zip folder for remaining XML file. You need to copy the XML text from zip folder and update on relevant XML text on dev.islg using admin side (in XML agreement).
After completion of following step, Please check that XML file on dev.islg (under Treaty & Rules).
Please let me know your feedback.
Irit will test it and report the result here on Thursday.
Thanks,
Paul
Thanks,
Irit
BIT/0048 Italy-Morocco [Italian]
29-AUGUST screen capture after copying and pasting XML text:
BIT/0068 Philippines - Switzerland BIT (1997) [French]
29-AUGUST screen capture after copying and pasting XML text:
BIT/0104 Switzerland - Uzbekistan BIT (1993) [French Translation]
29-AUGUST screen capture after copying and pasting XML text:
More follow up with regards to highlighted Article Citator links (indicating the document count) to come.
Irit
BIT/0065 Pakistan-Switzerland [French]
BIT/0811 Belgium - Luxembourg - Poland BIT (1987) [French]
Another issue I want to mention (which may only be an issue on dev.islg) is that when we open the pinpoint paragraphs, the PDF documents do not open. See screen captures, using BIT/0811 as an example.
Thanks,
Irit
Thank you for reporting the results above.
Thanks,
Morgan
We are looking into above changes and will update you soon.
Morgan
Following, Both issues have resolved on dev.islg. Please check and confirm.
1) PDF Open issue
2) The count number [] place
Morgan
1) PDF Open issue has been fixed.
he pinpoint paragraphs I have tested so far are all working. Meaning, the PDFs are opening and directing me to the correct paragraph number.
I hope this helps.
Thank you,
Irit
The count number [] place issue has been resolved on dev.islg.
Now, For heading it has been displayed after Article Number and for other cases it has been displayed before the sub-paragraphs.
Please check and confirm.
Thanks,
Morgan
The issue with the count number [] placement has been fixed. Every document I have checked so far looks great.
Issue resolved
BIT/0811 Belgium - Luxembourg - Poland BIT (1987) [French]
4-September screen capture:
Thanks,
Irit
Looks like we finally have the issue resolved.
Please confirm, and we'll implement the fix on our end.
Thanks,
Morgan
We will replace XML files with above attached linknum_xmlfiles.zip files on production environment. Once it will be done You need to again do same process which we did on dev.islg.
You need to replace XML text (by using attached XML files) through production admin side.
Let me know if you have any query.
Scheduling the to-do for next Tuesday.
Thanks,
Morgan
Yes, I'll make myself a note to that next Tuesday morning.
Thanks,
Irit
Thanks,
Morgan
We have uploaded XML files on www.islg. Please do same process which you did on dev.islg. Just update the XML text on www.islg admin side.
To update XML text, Please take reference from above attached linknum_xml ZIP Files.
Morgan
I'll make the updates and report back.
Thanks,
Irit
Morgan
Could you provide an update on updating the XML text for the linknum_xml ZIP files. Has this been completed yet?
Also,
Thanks,
Morgan
Irit has to test one more XML before she can confirm whether it works fully or not (replacing XMLs by copying and pasting).
Other than that, all the identified issues seem to have been resolved now.
Thanks,
Paul
Morgan
The final XML has been confirmed. There are three issues I found while updating.
In the two XMLs below, BIT/0141 and BIT/0967, the paragraph numbers that have a hyphen are getting duplicated (or are repeating). For example, "1.-" has become "1.-1.-"
BIT/0141 El Salvador - Spain BIT (1995) [Spanish]
Link to Admin site:
https://www.investorstatelawguide.com/documents/documents/BIT-0141%20-%20El%20Salvador-Spain%20BIT%20(1995)%20[spanish]%20UNTS.pdf
PDF:
Link to Live site:
https://www.investorstatelawguide.com/Treaties/AnnotDocument?toc=annotSection&agreementID=280&tabcontent=&cat=bit
XML:
BIT/0967 Cape Verde - Portugal BIT (1990) [Portuguese]
Link to Admin site:
https://www.investorstatelawguide.com/documents/documents/BIT-0967%20-%20Cape%20Verde-Portugal%20(1990)%20[Portuguese]%20.pdf
PDF:
Link to Live site:
XML:
https://www.investorstatelawguide.com/Treaties/AnnotDocument?toc=annotSection&agreementID=1795&tabcontent=&cat=bit
BIT/0099 Belgium - Luxembourg - South Africa BIT (1998)
On the Live site, the paragraph spacing and Table of Contents don't look right. I'm not sure why there's an issue with the XML.
Link to Admin site:
https://www.investorstatelawguide.com/documents/documents/BIT-0099%20-%20Belgium-Luxembourg-South%20Africa%20BIT%20(1998)%20UNTS.pdf
PDF
Link to Live site:
https://www.investorstatelawguide.com/Treaties/AnnotDocument?toc=annotSection&agreementID=258&tabcontent=&cat=bit
XML
XML - Table of Contents
Thank you,
Irit
Please provide the original XML files of following XMLs.
1) BIT/0141 El Salvador - Spain BIT (1995) [Spanish]
2) BIT/0967 Cape Verde - Portugal BIT (1990) [Portuguese]
3) BIT/0099 Belgium - Luxembourg - South Africa BIT (1998)
I have attached the original XML files for:
1) BIT/0141
2) BIT/0967
3) BIT/0099
Thank you,
Irit
We have resolved following 2 files issue on www.islg.
For third file, there is spacing issue which we need to treat separately. We will do further process on 3rd file and update you soon.
Morgan
I checked BIT/0141 and BIT/0967. The issue has been resolved for these two.
Thank you,
Irit
The 3rd File (BIT/0099 Belgium - Luxembourg - South Africa BIT (1998))' s spacing issue has resolved and uploaded on www.islg.
Please check and confirm.
I checked BIT/0099 Belgium - Luxembourg - South Africa BIT (1998)
On the Live site, the paragraph spacing and Table of Contents issue has been fixed. However, now there are foreign characters appearing throughout the XML.
Link to Live site:
https://www.investorstatelawguide.com/Treaties/AnnotDocument?toc=annotSection&agreementID=258&tabcontent=&cat=bit
I have resolved the foreign character issue on above file. Please check and confirm.
I checked BIT/0099 again and it looks great.
Thank you,
Irit
Morgan
On www.islg, the issues found in BIT/0141, BIT/0967 and BIT/0099 have all been resolved.
On dev.islg, the issues found in BIT/0141 and BIT/0099 are still there.
Irit
Thanks,
Morgan
We have also resolved above issue on dev.islg. Please check and confirm.
I checked dev.islg. The issues found in BIT/0141 and BIT/0099 have been resolved. Morgan, I will mark this to-do as complete.
Thank you,
Irit