Ability to extract a Provision List directly from HTML document

Hi Harsh ,

Attached are two short Treaty documents: BIT/1118 (10 pages) and BIT/1121 (11 pages)

BIT-1118 - Libya-Switzerland (2003) [English].pdf 362 KB • Download

BIT-1121 - Gambia-Switzerland (1993) [English].pdf 1.2 MB • Download

Thanks Irit .

Do we have another that is better quality? I'm concerned these will have issues with the automated converter when Harsh attempts to create a sample HTML.

Thanks,

Morgan

Hi Harsh , please see the following PDF samples.  They are better quality. BIT/0916 and BIT/0917.

BIT-0916- Italy-Mongolia (1993) [English].pdf 139 KB • Download

BIT-0917- Netherlands-Nicaragua (2000) [English].pdf 27.4 KB • Download

Great. Thanks Irit !

Hi Morgan , Martin and Irit ,

I have uploaded one sample Treaty PDF document on staging.islg and converted into html through our application so the provisions are automatically added into database.

Now, we need to provide this same html document to CDS team so they can make further Treaty & Arbitration html by using this sample html file as reference.

You can forward this attached html file to CDS team to take this html as reference.

ARB-0012 - ICSID Arbitration Rules (2012).html 120 KB • Download

ARB-0012 - ICSID Arbitration Rules (2012).pdf 139 KB • Download

Great. Thank Harsh .

However, I noticed that the HTML document above does not have all the structure we want it to. For example, Chapter I is created as a standalone <div> tag and does not indicate that Rules 1-12 are all contained within Chapter I:

image.png 235 KB • Download

Martin and Harsh , I assume this is something that CDS could manually integrate into the HTML documents when they code them?

Thanks,

Morgan

Yes Morgan , you are correct. The class name Provision is important to make in div element. for example  if there is any div element then we apply class="provision and div tag Id as per following example.

Rule 1 is div tag in html then  class="provision" and id need to apply for Rule 1 div tag ( <div data-pro='RULE2' data-key='32' class='para paralvl1 provision' id='BIT1365_paRULE2.1'>)

I will discuss with Martin in today's call so he can guide or discuss with CDS team.

Hi Irit ,

Can you ask CDS to start converting the 2 documents you have posted in this thread?
They should use the following items to guide them:

While CDS works on converting the documents, Harsh will continue coding ISLG Admin so it can extract the Provision List from a manually converted HTML doc. Our goal is to then use the documents manually coded by CDS to test against the new code.

Could CDS turn these documents within 1 week (or less)?

Also note that after this project is complete, we will probably need to make small adjustments to our HTML coding manual.

Thanks,
--Martin

Hi Martin ,

Yes, I will send an email to CDS today.  I'll let them know this is high priority and we need this back within a week or less as you suggested. 

Thanks,
Irit

Thanks Irit . Could you please cc Martin and myself on the email to CDS.

Morgan 

Morgan , yes.  An email was sent to CDS today.

Thanks,
Irit

Good morning Harsh   Morgan Martin ,

CDS has completed the HTMLs for samples, BIT/0916 and BIT/0917:

BIT-0916- Italy-Mongolia (1993) [English].html - TOLOGIX - PDF to HTML Conversion

BIT-0917- Netherlands-Nicaragua (2000) [English].html - TOLOGIX - PDF to HTML Conversion

Thank you,
Irit

That's great. Thanks Irit .

Harsh , I assume these will work in providing you the sample documents you need?

Morgan

Hi Morgan , Irit   and Martin ,

We checked the html and found that there is spelling mistake in html for provisionDefinition class.

As per following screenshot, there is spelling mistake.

provision.png 157 KB • Download

In Red box, you can see the spelling of provisionDefination.

Please ask to CDS team to correct the spelling in both html and revert back to us.

The correct word is : provisionDefination

The provisionDefination is use for  fetch the title of provision.

Hi Morgan and Irit ,

Harsh and I had a discussion this morning. For these 2 documents, Harsh will make the changes himself (so there will be no need to have CDS do them), but we will also need to inform CDS of the class name change.

There is a typo in "provisionDefination", but this is the way it was coded in our code, so unfortunately this is what CDS will need to use moving forward.

Harsh will be updating the HTML Coding Manual to add a new "Provision" part and explain this.

Morgan : can you confirm that the latest copy of the HTML Coding Manual is the one I have attached?

ISLG HTML Coding Manual (21-feb-20).docx 1.41 MB • Download

Thanks,
--Martin

Hi Morgan and Irit ,

Today, We have deployed this task on staging.islg. You can check and provide your feedback.

When you select Edit Provision from Dropdown menu then you can get button for Provisions.

You can use following 2 attached html files for fetch the provisions. As per Martin above comment, We replace the spell and update the following 2 HTMLs.

BIT-0916- Italy-Mongolia (1993) [English].html 43.9 KB • Download

BIT-0917- Netherlands-Nicaragua (2000) [English].html 35.6 KB • Download

 

Hi Martin ​,

Yes, that's the latest version of the coding manual. Note all the coding manual versions are contained here: https://islg.egnyte.com/fl/9VgmHKWgaz.

Does this mean we're stuck with that typo?

Thanks,

Morgan

Great. Thanks Harsh ​.

Irit ​, could you please test this on staging and report back the results.

Thanks,

Morgan

Hi Morgan ,

We are stuck with this typo for the time being. Harsh and I discussed the effort required to make the change. While the effort is not huge, several parts will be affected.
One of these parts is the PDF to HTML conversion engine. Since we are in the midst of upgrading our conversion engine, I think the best approach is to make that change at the time when we do the official switch to the new engine.

Thanks,
-Martin

Ok. Sounds good, Martin ​. Makes sense.

Morgan

Hi Morgan , Martin Harsh  

Success. I uploaded the HTMLs for BIT/0916 and BIT/0917 on staging.islg. I then selected the Get Provision button.  The provision list was pulled from the HTML. 

Please let me know if you would like for me to confirm this on DevOps and if I need to change the state to “Tested on Staging”.  

Thank you,
Irit

Hi Irit ,

Great! Yes, please change status to "Tested on Staging" if you are satisfied with the functionality.

Thanks,
--Martin

Hi Martin ,

Here, I have updated user manual for Treaty and Arbitration html documents. You can find the update in second page. I marked out as grey color.

ISLG HTML Coding Manual (29-Oct-2021).docx 1.42 MB • Download

you can update the text or content if you want.

Hi Harsh , thanks. I reviewed the changes and they look good to me.

Hi Irit : can you share the latest document with CDS? Note that I have also uploaded a copy to our Egnyte share (\Shared\ISLG\Database Development & Maintenance\ISLG Rebuild\HTML Coding).
Thanks!

CC: Morgan  

Hi Martin ,

Will do, thanks.
Irit

Perfect. Thanks everyone!

Morgan