Analyze and Tag Template
Morgan
The Analyze and Tag documents template should be ready for review here:
http://tologix-islg-rebuild.s3-website.ca-central-1.amazonaws.com/analyze-and-tag.html
TOLOGIX - ISLG App Rebuild
Morgan
The Analyze and Tag documents template should be ready for review here:
http://tologix-islg-rebuild.s3-website.ca-central-1.amazonaws.com/analyze-and-tag.html
The templates looks good. I've noticed a few issues detailed below. I'm not sure if these will be dealt with by the development team when they implement the templates, but I want to ensure they are addressed as necessary here:
Morgan
1: I added an example of two colours overlapping
2: The highlighted text will now become unhighlighted when clicking on copy text. (the focus outline will remain)
3: Please clear your temporary browser cache (CMD+SHIFT+R) this appears to be working as expected.
4: There is an HTML area set up for each of the tools however I only populated the Article Citator as an example. The others can be populated using existing components similar to the article citator tab.
Morgan
Thanks for providing HTML.
We have found some minor bugs in html.
1) If we select two different paragraphs then whole text it is going to merge in one single paragraph.
2) if we select paragraph text and then after click on selected text the selected text formatting is disturbing. It is going to overlapping the text.
In addition to the issues
Morgan
An update on this - Kevin is working on the customizing the library and will have this completed by mid next week.
Thanks
Savannah
Morgan
Monday, September 9 at 8:30 am
Tuesday, September 10 at 10am (during our weekly Tologix call with Ryan and Mitch)
Savannah
The bug fixes are available here: http://tologix-islg-rebuild.s3-website.ca-central-1.amazonaws.com/analyze-and-tag.html
If you're not seeing the fixes be sure to clear your browser cache.
The other tabs currently have a placeholder string where the dynamic forms should be loaded:
The tool select box will switch tabs and data attributes on the excerpt to reflect the tool change.
Updated assets can be found here:
The scripting and HTML managing this interface is quite complex. Please let us know if you run into any problems.
Items #2 and #3 that identified above appear to be working well now. However, for item #1 the except remains highlighted after selecting "Copy Text".
Two other issues to address:
Morgan
The copy text issue should be fixed.
1. Now in cases where the modals would extend outside the left margin they will now align flush with the left margin.
2. I have added logic to close modals when clicking outside of them.
3. Filtering logic is part of the "review" workflow and should be managed server side during app development.
Understood on item #3.
Thanks,
Morgan
Thanks for providing html.
The Integration of Anyzing/Tag html is going on. We will let you know if we find any difficulty.
Could you please let me know where you use above annotator.min.cs ? Does we need to add annotator.min.cs in our application ?
No sorry, annotator.min is not in use. I had included it when i was brainstorming a solution but ended up going with a fully custom approach instead.
We have integrated HTML in our application but we are in trouble at one point. Your static HTML structure and our dynamic generated HTML structure are different.
For Example, You have taken HTML with only <P> tag but our html has been generated <div> base. So may be, Because of that reason, The Tag Tool
(AC, JC, TP,) is not generating at proper place.
Let me know once you come in office. We will discuss with Skype Call.
What is your Skype ID ? I will ping you
Following, I have mentioned the HTML link which provide you dynamic generated html. Please take reference of that html and do necessary changes.
Also, Please provide us only the portion which you will change in both JS and CSS.
http://dev.convertortool.com/HTML/ADM-CPI%20v%20Mexico.html
That said using this should at least greatly improve the issue:
.document *{
}
When text is properly formatted with single wrapping elements you will get results like this:
However messy nesting will have results like this:
Thanks
I'd prefer the resolve this somehow if we can, but if not we can work with this given that it will be very rare to use these sections of the documents for excerpts.
Morgan
In order to get clean results in output of the Tool bubbles and line indicators we need to have a single identifiable element wrapping every block of text.
The first screenshot above is an example of that working correctly for us as the formatting is clean enough to make it work.
However the second screenshot is really messy. I'll do my best to explain:
This one line of text is wrapped in many <div> elements with inline styles. but the entire block of text including the lines above/below are not wrapped in a single common element for me to target.
In the first screenshot where it is working with my tagging implementation looks like this:
1 <div> element wrapping the entire block of text.
Note that I've added
Thanks,
Morgan
1) 2 Column Structure HTML: Where we need to separate 2 <div> elements in Main <div> element.
2) Sub Paragraph Structure : Where we need to created sub-paragraph <div>(red color) nested under Main Paragraph <div> (orange color).
For example, in the above screenshot the divs that have width:50%; should also have the paraText class.
However because of the way the divs are formatted in the columns there will never be a way to highlight and tag just the left block of text or just the right block of text because each line of text is inside its own div. The only way to fix that would be to wrap the entire left column in one identifier, and the entire right column in one identifier.
Will do one thing in algorithm. We will set one common class="Paratag" for all elements where text is wrapping.
You can use that "Paratag" class for identify the html element.
Does it work for you ??
One more thing we noticed in your HTML.
If we change the html structure as per attached screenshot. It means, If any sub-paragraph or bullet point is started inside main paragraph that time it is creating issues to display excerpts.
Please look into the following screen shot.
What you're seeing in the screenshot above is expected behaviour
indicators are aligned to the paragraphs in which the excerpts exist.
In order for the nesting to work, we would need <div class="paraTag"> wrapping everywhere there is a red box in these images:
Only for two column structure, We are not able to give <div class="Paratag"> to whole box.
We can able to apply to class whole horizontal line or half line as per following screenshot.
Let me know.
If that's the case then we're not going to be able to resolve issues in those sections of documents.
I think we need to bring this into a broader discussion about how the HTML documents are getting structured. Therefore, let's prioritize getting the document prepared as we discussed above.
Note it is really concerns me that we can't identify the appropriate paragraphs in the document above given that the original PDF was structured appropriately as evidenced by the excerpt I highlighted in the screenshot below. If the original PDF has the appropriate structure to identify the paragraph, why wasn't this picked up in the converted HTML document?
Thanks,
Morgan
We are using "ByteScoute" 3rd Party tool for read the text from PDF. This tool is providing text line by line.
In Two Column Structure, We need to divide one whole line in 2 portion. Because of that, We need to put first half line in one <div> elemnet and second half line in second <div> element. (Refer following screenshot)
Currently, As we discussed with Kevin, We are only facing excerpt problem in 2 column structure. Rather than 2 column structure we are able to set <div class="Paratag"> as common element.
Further more, We are communicate with our UI team to convert 2 separate <div> element in to one single <div> element to resolve this problem.
Team is working on it to resolve this as soon as possible.
We will keep you updated.
As I've mentioned above, it would be ideal to represent the text in the same column as the same paragraph (both for excepts purposes, and to properly represent the text), so let me know once the UI team has been able to take a closer look.
Thanks,
Morgan
I just got off a call with
As a starting point,
The document will ensure everyone has a mutual understanding of how the HTML code is structured, and address any issues that may affect how we use the documents within the applications. It will also serve as a precursor to creating a more formal HTML coding manual.
Please let me know if you have any questions or concerns.
Thanks,
Morgan
Sure, we will do it. Let us know how we make it easier as possible to understand html document. It is important at this point so we all in same page and get clarity of ongoing development that full fill your requirements.
Thanks,
Jitesh
We'll get back to you soon with all the detail we'll need in the document.
Morgan
As we discussed above, The Team is working on to resolve two structure column selection issue and applying <div class="paratag"> to all places where text is wrapping in element.
Probably, It will be done by tomorrow.
We will do our best to provide updated html structure before end of tomorrow.
Thanks,
Morgan
Thanks!
Ryan
Morgan
Jitesh has posted HTML Document structure in relevant basecamp conversation.
Further more, As we discussed, Here I have attached updated sample HTML file where you will find <div class="paratag"> and also it is allowing to select 2 column structure.
However, further to the screenshot below it appears that footnotes are getting inserted within the last <div class="paratag"> on the PDF page. This needs to be changed. Footnotes should be separated from the body text of the page (except for the <span class="supertext" linkid="_fntag"> that connects the relevant footnote to the body text).
Also, I found a similar problem with some of the headings. In the screenshot below, a heading is included in the previous paragraph. I understand how the conversion algorithm might interpret the heading as a sub-paragraph, but is there a way for us to prevent this from happening by identify the text as a heading based on the text font (e.g., bold, underlined, larger font, etc.)?
Thanks,
Morgan
We found one issue in HTML template. If we select multiple paragraphs then one paragraph has not selected. Please refer following video for more clarification.
one small typo in the script may be causing this. Should be fixed with this file.
Could you pleaese provide only updated code? Because it is difficult for us to find that change or replace whole js.
Are you making changes directly within the minified javascript file I provided?
If so that's going to get extremely difficult for me to debug issues you're having going forward. This is why I suggested making your own file separate from mine since the beginning.
In my file the change looks like this. The highlighted quote was missing.
$(this).wrapInner('<span class="tagged__excerpt" data-excerpt="' + newid + '"></span>');
Thanks for information.
Sorry for late reply. Actually,Yesterday I was on leave.
Actually, We need to do some changes in minified JS for dynamic Datatable rendering and some other function which conflicted in our structure.
We are not changing your code. We are just adding our relevant code in your function.
We have added " in tagged_excerpt which you mentioned above. but still we are facing same issue.
We are not able to highlighted multiple paragraphs.
Here, I have attached our sassquatch.min.js file for your reference.
I'm sorry, I am unable to debug a minified JS file. This is why I requested all custom Javascript you write be in a standalone file separate from the javascript I provided. The example I have here is working as expected:
http://tologix-islg-rebuild.s3-website.ca-central-1.amazonaws.com/analyze-and-tag.html
When I write my scripts I organize them by functionality and minify them into a single file at the end.
It's important that we're not making modifications directly to the modified scripts as the minification tool renders the functions ineligible.
The unminifed scripts are as follows:
These get compiled into a single file:
OR
Following, I have attached the HTML file which shown in video and I will take reference of sassquatch.min.js for do this change in our application.
Hope this is fine.
I have checked above mentioned URL and seems like issue is resolved now.
Please let us know when you will be completed your testing. So, Tomorrow we will integrate this change in our application.
in your JS on line 304
change :
n(this).find("> p").each
to:
n(this).find("> *").each
This should drastically improve the issue outlined above.
In Analyzing/Tagging, We are facing challenges while loading saved Tags.It is not highlight selected Tag text by using ranges at load time of HTML File which already tagged.
In your demo HTML, There is Tags saved in static HTML file itself but we could not save Tags directly into HTML file, so creating Tags Run-time inside of Tag there is challenge of fetching Range offset (startOffset,endOffset) because JS method referencing most parent element that is span tag created run-time while we tag some text and highlight text,
so can you help us to load Tag run-time using Ranges. we are asking you because it is difficult to find exact offset of second tag text inside outer tag text.
Thanks.
I'm working on writing a function to generate saved tagged excerpts on page load, however it would be helpful to have a sample of the data you're trying to use to generate tags on page load.
The UI for creating the tagged excerpts on the page does not have context as to the actual tags on the excerpt. You should be using a unique ID on the tagged excerpt which is used to map the excerpt to the tags that are attached.
I won't be able to write the entire functionality to handle this for you.
I think the easiest way of going about this would be to save the HTML of the document as it appears after tagging the document rather than generating all the tags on load.
The UI for creating the tagged excerpts on the page does not have context as to the actual tags on the excerpt. You should be using a unique ID on the tagged excerpt which is used to map the excerpt to the tags that are attached.
we have already set unique ID run-time and based on that ID we have save our Excerpt into Database and we can load our Excerpt run time for single Tag but there is difficulty to load Tag into another Tag.
I think the easiest way of going about this would be to save the HTML of the document as it appears after tagging the document rather than generating all the tags on load.
We have already looked into this way, and we found that if we save our Excerpts into HTML file directly then migration of old data is very difficult to migrate into new application. Hence we don't have old Tag data into new HTML file.
Thanks.
I don't know what your old data looks like and without that there is nothing I can do. If the only information I have to go on is a startOffset and endOffset it's not going to be possible to generate the tags properly on runtime with javascript.
The way it would have to be done in javascript requires looping over the data and wrapping the text in HTML tags programatically. The problem is every time a new HTML tag is rendered the offsets would change and would no longer be highlighted correctly.
I think we need to take a call to discuss this issue broader way and also need to discuss about how we migrate existing islg data in our new tagged feature.
Let me know your thoughts and will do call on next week.
Morgan
We need to have a discussion internally about this issue at Industrial first.
Morgan
Thanks,
Morgan
Tomorrow, We have holiday in DEVIT.
Can we reschedule call on Wednesday (9th October) at same time ?
Wednesday at the same time works for me.
Morgan
Mel, Kev and I are unavailable on Wednesday. I can invite Kev to our biweekly meeting on Thursday?
Savannah
Ok, but let's move up the meeting to 745am Vancouver time, and we'll make this the first item on the agenda. That way
Thanks,
Morgan
We can do 8am Vancouver time, Mel, Guillaume and Kev are all in a meeting until then.
Savannah
Morgan
As per your following code in Sasquatch.min.js, The multiple highlighted paragraph is taking whole page selection range. Actually we need only particular selected paragraph range.
"" != t.toString() && (window.getSelection(), function () {
var t = function (t) {
var e = t.commonAncestorContainer,
a = new Array(0),
o = new Array(0);
if (t.startContainer != e)
for (var s = t.startContainer; s != e; s = s.parentNode) a.push(s);
if (0 < a.length)
for (var s = 0; s < a.length; s++) {
var n = document.createRange();
s ? (n.setStartAfter(a[s - 1]), n.setEndAfter(a[s].lastChild)) : (n.setStart(a[s], t.startOffset), n.setEndAfter(a[s].nodeType == Node.TEXT_NODE ? a[s] : a[s].lastChild)), o.push
}
var i = new Array(0),
r = new Array(0);
if (t.endContainer != e)
for (var s = t.endContainer; s != e; s = s.parentNode) i.push(s);
if (0 < i.length)
for (var s = 0; s < i.length; s++) {
var d = document.createRange();
s ? (d.setStartBefore(i[s].firstChild), d.setEndBefore(i[s - 1])) : (d.setStartBefore(i[s].nodeType == Node.TEXT_NODE ? i[s] : i[s].firstChild), d.setEnd(i[s], t.endOffset)), r.unshift(d)
} {
if (!(0 < a.length && 0 < i.length)) return [t];
var l = document.createRange();
l.setStartAfter(a[a.length - 1]), l.setEndBefore(i[i.length - 1])
}
return o.push(l), response = o.concat(r), response
}(window.getSelection().getRangeAt(0));
d++;
for (var e = 0; e < t.length; e++) a(t[e], e)
}())
Is it possible for you to do this change to select only paragraph range rather than whole page ?
Morgan
I apologize for the delay on my response. I'm not really understanding what the issue is here.
I suggest you setup a call to discuss and resolve the issue.
Thanks,
Morgan
Can we do call on Thursday 8:30 AM (Ottawa Time) ?
Unfortunately Kevin is not available then. Are you free at 10:30 Ottawa time?
Please confirm that We are going to take call on 10:30 AM Ottawa time today.
Skype Id : harsh.parikh05
the zoom details are below:
Industrial is inviting you to a scheduled Zoom meeting.
Join Zoom Meeting
https://zoom.us/j/361713802?pwd=QllVSHM0Q1ZtMFlvYTk4REVZRmRBUT09
Meeting ID: 361 713 802
Password: 067965
One tap mobile
+16475580588,,361713802# Canada
Dial by your location
+1 647 558 0588 Canada
+1 646 558 8656 US (New York)
+1 720 707 2699 US (Denver)
Meeting ID: 361 713 802
Find your local number: https://zoom.us/u/abTvh73RhC
Here is our understanding of the problem:
Currently, the tags are based on the document range. But you would like the tags to start and finish based on paragraph range.
We have provided markup for the templates, but to resolve the issue will require more in depth understanding of the technical requirements.
From our perspective, this needs to be handled within the application development and not template development. We suggest having a call with the whole team to come to a solution.
We do have a call scheduled for next Thursday, but ideally this should be worked out sooner. Let us know when you are available next week to discuss.
Savannah
Morgan
Sounds good. We can make tomorrow at 7:30 work. We can't do right at 9am on Monday as we have an off-site meeting, but could do 9:30.
Savannah
Morgan
Monday work from our side, please do schedule on Monday 9:30 am.
Thanks,
Jitesh
In last call, You had asked approach document for Analyzing/Tagging which we are going to implement in order to achieve current and new ISLG tagged/cited data. We have prepared document describe how technically we will highlight tagged/cited data in Analyzing/Tagging for both current and new ISLG.
Please find attachment of Analyzing/Tagging approach document.
cc:
Thanks,
Jitesh
Thank for this.
Thanks,
Morgan
We have reviewed the above document and have some concerns about the suggested approach to document tagging which we'd like to discuss tomorrow:
- This approach still requires client side tag rendering which will negatively impact performance and is too heavy a task for browsers to handle which may result in users browsers crashing
- We don't understand why the paragraphs are being cloned. Is this only used to aid migration? How does it help? This adds significant complexity to the logic required for creating the tag markup
- How will this accommodate tags across paragraphs?
Thanks
Savannah
We'll discuss more during the call tomorrow, but let's ensure we coming are to the discussion fully informed, and prepared to offer solutions.
Thanks,
Morgan
This approach still requires client side tag rendering which will negatively impact performance and is too heavy a task for browsers to handle which may result in users browsers crashing
Ans : Yes, It is client side tag rendering but it will not impact negatively because it is not rendering any extra resources by client side like files, images or ajax call. it will simply clone those text (paragraph) area which are cited (tagged) and display highlighted as per your requirements. We are not doing clone any extra text or whole document to reduce performance or browser crashing. It will simple clone only selected text on which admin had cited.
- We don't understand why the paragraphs are being cloned. Is this only used to aid migration? How does it help? This adds significant complexity to the logic required for creating the tag markup
- How will this accommodate tags across paragraphs?
Ans : It is used to aid migration as well as new data. By this approach we can also highlight old data which are already cited in current application and obviously new application as well.
Thanks,
Jitesh
Following up on our discussions this week, please let us know when you have a prototype of the Analyzing/Tagging tools available for
Thanks,
Morgan
Tomorrow, We will provide you one test document in Analyze Data Capture task on rebuild.islg (Demo URL) and also provide you step to how to open the document to review tagging portion.
Because, We need to do lots of thing to re-create to prepare separate prototype and upload new prototype on server.
Following, We have mentioned the URL and step to how you will reach the document for Analyze/Tag.
URL : http://rebuild.investorstatelawguide.com/admin
User Name : Admin
Password : Admin@123
After successfully login, Please do following setp.
1 - Click on Analysis in Progress: Data Capture link from left side menu under Document State menu
2- Click on Document name
3- Now, You are able to select any paragraph and look how the Analyze/Tag is currently working now.
I have refactored the tagging logic to target the .para elements rather than the .document. I also created a new function that you can call from anywhere to get the current start/end offsets.
At any point you can use the variables 'startElement' or 'endElement' which are storing the start/end html element ID's of the selected range.
A working example of these variables and functions can be found at http://tologix-islg-rebuild.s3-website.ca-central-1.amazonaws.com/analyze-and-tag.html. Just make a selection and the start/end offsets will be displayed in an alert.
In tagging.js on line 102 you will find where I'm calling these example functions.
Start Offset
(paragraph-11 is what's stored in the variable 'startElement')
(175 is the offset returned by the function):
End Offset
(paragraph-13 is what's stored in the variable 'endElement')
(182 is the offset returned by the function):
Morgan
Morgan
Could you provide us with an update on whether the issues above have been resolved, and whether there is anything we need to discuss during the team call on Friday.
Thanks,
Morgan
The above issue has resolved and we integrated in our application. Technically, currently we are facing only following one issue.
Morgan
I sent a message on BC to schedule that call for next week here: Tagging - Knowledge Transfer - TOLOGIX - ISLG App Rebuild
We will update you on the go forward plan after the call.
Savannah
We are available for any day. Please let us know your convenient day and time.
Thanks,
Jitesh
I sent you an invite for Tuesday, Jan 21 at 11am.
cc
Also, my understanding is that
Thanks,
Morgan
Thanks,
Morgan
I understand that you had your call this morning with
Thanks,
Morgan
Yes, We had call with Kevin Andrews Kevin and Savannah Mitchell Savannah and they are asked me to resolve current issue of multiple paragraph selection from DevIT so we will work on it and let you know as soon as possible. Thanks.
Thanks,
Jitesh
We are on going to resolved issue of multiple paragraph selection and most of cases we have resolved and our QA is doing testing it.
Thanks,
Jitesh
Thanks,
Morgan
Thanks for the update, this is great news!
To confirm, we will not be reviewing this feature or providing feedback until the cards are in UAT.
Mel
Thanks,
Jitesh
Thanks,
Morgan