Skip to content

Commit

Permalink
Revised some properties of some markers within the AI_USFM.xml contro…
Browse files Browse the repository at this point in the history 
…l file. Changed the \xt marker to have userCanSetFilter="1" since can be a stand-alone marker that can be filtered. Hence, \xt now has its own entry in the Doc's filter list in the UsfmFilteringPage. Also set the inform="1" and navigationText to a suitable abbreviation for the table-related markers that previously had inform="0". This helps to identify those markers for texts that have substantial numbers of empty markers.

Revised the filtering apparatus in the filterPage to allow for the \xt marker to serve its dual purpose in Usfm3 - that of an embedded marker within an \x ...\x* span and also as a stand-alone attribute marker that requires a vertical bar to delimit hidden information. This also required some changes to the App's DoUsfmFilterChanges() function to enable the \xt markers double role. It also required changes to some helpers.cpp functions including the RemoveFilterMarkerFromString() and AddFilterMarkerToString().
For better organization, a few fast-access strings that were held on the Doc have been moved to the App to join the group of other fase access markers. When they were on the Doc, there were not updated properly with certain markers they should have had.
To get the unfiltering code to work properly I needed to remove the if (bIsFirstNode) block from the bUnfilteringRequired block in the Doc.
Also, I reinstated a block of code - with revisions - to remove filtered info from the last word of a merged source phrase when the filtered information on the top level source phrase is unfiltered.
The previous code was able to filter and unfilter the fig marker, but wasn't able to handle adequately the other attribute markers such as \jmp, and free-standing \xt that also contain bar delimited data. That issue has been fixed, but there is still more work to do before the filtering routines can re-filter some attribute markers after previously unfiltering them.
  • Loading branch information
@pngbill
pngbill committed Mar 4, 2024
1 parent 0ccb2a1 commit a776754
Show file tree
Hide file tree
Showing 9 changed files with 1,710 additions and 1,007 deletions.
1,006 changes: 616 additions & 390 deletions source/Adapt_It.cpp
View file

Large diffs are not rendered by default.

38 changes: 32 additions & 6 deletions source/Adapt_It.h
View file
Original file line number Diff line number Diff line change
Expand Up @@ -3092,6 +3092,14 @@ class CAdapt_ItApp : public wxApp
// was-current document after reading the other documents for the join
// operation. (If TRUE, document loading does nothing except set up the
// list of CSourcePhrase instances.)

// whm 29Feb2024 added the following wxString values to track which filtering
// markers were changed (selected/unselected) during a session in the USFM and
// Filtering tab of preferences.
wxString markersChangedToBeFiltered;
wxString markersChangedToBeUnfiltered;



// whm 23Aug2021 added the following for AutoCorrect feature support
// The following hash map is used to map typed sub-strings of target text to
Expand Down Expand Up @@ -4945,7 +4953,8 @@ class CAdapt_ItApp : public wxApp
// inline, but which do not bind more closely than punctuation
// (these are \wj \qt \sls \tl \fig; and their endmarkers are a separate
// wxString member -- see next line)
bool m_bIsEmbeddedJmpMkr; // BEW 13Mar20 added - to differentiate \+jmp from \jmp behaviours
// whm w23Feb2024 renamed m_bIsEmbeddedJmpMkr below to m_bIsEmbeddedFigOrJmpMkr
bool m_bIsEmbeddedFigOrJmpMkr; // BEW 13Mar20 added - to differentiate \+jmp from \jmp and \+fig from \fig behaviours
// (see extended comment regarding need for this, lines 15,600-15,635 of ParsePreWord())
wxString m_FootnoteMarkers;
wxString m_CrossReferenceMarkers;
Expand All @@ -4966,13 +4975,30 @@ class CAdapt_ItApp : public wxApp
bool FindProhibitiveBeginMkr(CSourcePhrase* pSrcPhrase, wxString& beginMkr); // BEW 11Apr20
bool FindProhibitiveEndMkr(CSourcePhrase* pSrcPhrase, wxString& endMkr); // BEW 11Apr20

bool m_bExt_ex_NotFiltered; //BEW 18Apr20, TRUE when unfiltered \ex ... \ex* is being parsed in
bool m_bExt_ef_NotFiltered; //BEW 18Apr20, TRUE when unfiltered \ef ... \ef* is being parsed in
bool m_bMkr_x_NotFiltered; //BEW 18Apr20, TRUE when unfiltered \x ... \x* is being parsed in
bool m_bUnfiltering_ef_Filtered; //BEW 18Apr20, TRUE when unfiltered filtered \ef ... \ef*
bool m_bUnfiltering_ex_NotFiltered; //BEW 18Apr20, TRUE when unfiltered filtered \ex ... \ex*
// whm 26Feb2024 removed the m_bExt_ex_NotFiltered and m_bMkr_x_NotFiltered globals below.
// The IsAttributeMarker() now check the gCurrentFilterMarkers directly for the filter status.
//bool m_bExt_ex_NotFiltered; //BEW 18Apr20, TRUE when unfiltered \ex ... \ex* is being parsed in
// whm 26Feb2024 The following m_bExt_ef_NotFiltered is set FALSE, but never used in code, so is now removed
//bool m_bExt_ef_NotFiltered; //BEW 18Apr20, TRUE when unfiltered \ef ... \ef* is being parsed in
//bool m_bMkr_x_NotFiltered; //BEW 18Apr20, TRUE when unfiltered \x ... \x* is being parsed in
// whm 26Feb2024 the following m_bUInfiltering_ef_Filtered is never used in code, so is now removed
//bool m_bUnfiltering_ef_Filtered; //BEW 18Apr20, TRUE when unfiltered filtered \ef ... \ef*
// whm 26Feb2024 the following m_bUnfiltering_ex_NotFiltered is set FALSE, but never used in code, so is now removed
//bool m_bUnfiltering_ex_NotFiltered; //BEW 18Apr20, TRUE when unfiltered filtered \ex ... \ex*

// whm 29Feb2024 added the following 2 booleans to track the status of the \x and \xt marker before
// a filtering session using the filter page in Preferences.
bool m_bMkr_x_WasFilteredBeforeFilteringChange;
bool m_bMkr_xt_WasFilteredBeforeFilteringChange;
bool m_bMkr_x_WasUnfilteredBeforeFilteringChange;
bool m_bMkr_xt_WasUnfilteredBeforeFilteringChange;

wxString m_chapterVerseAttrSpan; // BEW 18Apr20, store nearest earlier ch:vs ref for error message

// whm 22Feb2024 moved the following two here to the App from the Doc, and renaming them m_...
wxString m_charAttributeMkrs;
wxString m_charAttributeEndMkrs;

// BEW 22Apr20 marker and endmarker sets for the new way to handle text colouring & TextType
// propagation, and type-changing decisions
wxString m_RedBeginMarkers;
Expand Down
1,299 changes: 789 additions & 510 deletions source/Adapt_ItDoc.cpp
View file

Large diffs are not rendered by default.

19 changes: 12 additions & 7 deletions source/Adapt_ItDoc.h
View file
Original file line number Diff line number Diff line change
Expand Up @@ -608,7 +608,7 @@ bool bMarkAsDirty); // might want it instantly saveable
wxString& wordBuildersForPostWordLoc, wxString& spacelessPuncts); //BEW created 27Jan11
wxString SquirrelAwayMovedFormerPuncts(wxChar* ptr, wxChar* pEnd, wxString& spacelessPuncts); // BEW
// created 31Jan11, a helper for round tripping punctuation changes
bool m_bIsInFigSpan;
bool m_bIsInInlineNonbindingSpan;
int nFirstSequNumOfSpan; // used for \fig .... \fig* span

wxString m_currentUnfilterMkr; // used when unfiltering filtered content having attributes metadata
Expand Down Expand Up @@ -855,14 +855,18 @@ bool bMarkAsDirty); // might want it instantly saveable

protected:

wxChar* m_ptr;
wxChar* m_auxPtr; // an auxiliary ptr for use in scanning when changing
// whm 28Feb2024 removed the following wxChar* pointers - these are too dangerous to have
// floating around on the Doc class undefined outside specific functions.
//wxChar* m_ptr;
//wxChar* m_auxPtr; // an auxiliary ptr for use in scanning when changing
// m_ptr is not wanted
size_t m_nBeginMkrLen; // includes the trailing whitespace character (typically latin space)
size_t m_nEndMarkerLen; // includes the trailing asterisk
size_t m_nSpanLength; // the length of the span of characters to be skipped
size_t m_nBeginAttrMkrLen; // includes the trailing whitespace character (typically latin space)
size_t m_nEndAttrMarkerLen; // includes the trailing asterisk
size_t m_nVisibleTextSpanLength; // the length of the visible caption/word(s) text between
// the attribute marker and the bar char delimiter. whm 28Feb2024 added.
size_t m_nHiddenDataSpanLength; // the length of the span of characters to be skipped
// over in the input text stream, so that AI no longer "sees" the metadata while
// running its tokenizing function
// running its tokenizing function. Includes the bar char in the length.
wxString m_cachedAttributeData; // store everything from the first bar (strBar) to the
// end of the matching endmarker here. The CSourcePhrase which gets to
// store the hidden stuff cached here may not be the current one - it
Expand Down Expand Up @@ -898,6 +902,7 @@ bool bMarkAsDirty); // might want it instantly saveable
// set taking attributes, store here
wxString m_strAttrEndMkr; // if an end-marker is found which is one of the
// set taking attributes, store here
//
wxString m_strMatchedMkr; // after parsing a marker, store it here temporarily
int m_nCachePileSequNum; // BEW 20Sep22, the m_nSequNumber value for the pile on
// which to cache the attributes data (hiding it from sight)
Expand Down
35 changes: 35 additions & 0 deletions source/UsfmFilterPage.cpp
View file
Original file line number Diff line number Diff line change
Expand Up @@ -144,10 +144,21 @@ void CUsfmFilterPageCommon::DoSetDataAndPointers()
tempFilterMarkersBeforeEditDoc = gpApp->gCurrentFilterMarkers;
tempFilterMarkersAfterEditDoc = tempFilterMarkersBeforeEditDoc;

// whm 29Feb2024 added the following initializations to determine the filtered state of
// the \x and \xt markers BEFORE any filtering change made by the user during this session
// of the Filter page.
marker_X_wasFilteredBeforeThisChange = tempFilterMarkersBeforeEditDoc.Find(_T("\\x ")) != wxNOT_FOUND;
marker_XT_wasFilteredBeforeThisChange = tempFilterMarkersBeforeEditDoc.Find(_T("\\xt ")) != wxNOT_FOUND;

// fill the Project's local filter markers strings (before and after edit) from those on the App
tempFilterMarkersBeforeEditProj = gpApp->gProjectFilterMarkersForConfig;
tempFilterMarkersAfterEditProj = tempFilterMarkersBeforeEditProj;

// whm 29Feb2024 added the following wxString values to track which filtering
// markers were changed (selected/unselected) during a session in the USFM and
// Filtering tab of preferences.
tempMarkersChangedToBeFiltered.Empty(); // = gpApp->markersChangedToBeFiltered;
tempMarkersChangedToBeUnfiltered.Empty(); // = gpApp->markersChangedToBeUnfiltered;

// These initializations are done here in the constructor, rather than
// in OnInitDialog, because tempSfmSetAfterEditDoc and tempSfmSetAfterEditProj can be
Expand Down Expand Up @@ -176,6 +187,12 @@ void CUsfmFilterPageCommon::DoSetDataAndPointers()

void CUsfmFilterPageCommon::DoInit()
{
// whm 29Feb2024 added the following initializations.
// These strings on the App will contain any markers that are
// actually changed when
gpApp->markersChangedToBeFiltered.Empty();
gpApp->markersChangedToBeUnfiltered.Empty();

// get pointers to our controls
pListBoxSFMsDoc = (wxCheckListBox*)FindWindowById(IDC_LIST_SFMS);
wxASSERT(pListBoxSFMsDoc != NULL);
Expand Down Expand Up @@ -1291,6 +1308,22 @@ void CUsfmFilterPageCommon::DoBoxClickedIncludeOrFilterOutDoc(int lbItemIndex)
checkStr.Trim(FALSE); // trim left end
// extract the whole marker part
checkStr = checkStr.Mid(0,checkStr.Find(_T(' ')));

wxString augWholeMkr = checkStr + _T(" ");

if (pListBoxSFMsDoc->IsChecked(lbItemIndex))
{
// The checkbox item was checked, store the marker of that item in
// tempMarkersChangedToBeFiltered
tempMarkersChangedToBeFiltered += augWholeMkr;
}
else
{
// The checkbox item was unchecked, store the marker of that item in
// tempMarkersChangedToBeUnfiltered
tempMarkersChangedToBeUnfiltered += augWholeMkr;

}

// ensure that the flag stored in the flags array reflects the new visible state of the
// checkbox
Expand Down Expand Up @@ -3926,6 +3959,8 @@ void CUsfmFilterPagePrefs::OnOK(wxCommandEvent& WXUNUSED(event))
TRACE1(" m_currentUnknownMarkersStr = %s\n", pDoc->m_currentUnknownMarkersStr);
#endif

gpApp->markersChangedToBeFiltered = usfm_filterPgCommon.tempMarkersChangedToBeFiltered;
gpApp->markersChangedToBeUnfiltered = usfm_filterPgCommon.tempMarkersChangedToBeUnfiltered;

// there was no change made to the SFM set, so we only have to check for filtering changes, and
// do the reparse if changes had been made
Expand Down
13 changes: 12 additions & 1 deletion source/UsfmFilterPage.h
View file
Original file line number Diff line number Diff line change
Expand Up @@ -55,7 +55,7 @@ class CUsfmFilterPageCommon : public wxPanel
// the value of tempSfmSetAfterEditProj.
// Note: Factory default sfm set is UsfmOnly and can't be changed,
// so we don't need before and after edit variables for it

bool bSFMsetChanged;

bool bChangeFixedSpaceToRegularBeforeEdit;
Expand Down Expand Up @@ -86,6 +86,17 @@ class CUsfmFilterPageCommon : public wxPanel
wxString tempFilterMarkersBeforeEditDoc; // initialized to equal gpApp->gCurrentFilterMarkers in the constructor
wxString tempFilterMarkersAfterEditDoc; // initialized to equal tempFilterMarkersBeforeEditDoc in the constructor

// whm 29Feb2024 added the following values to track which filtering
// markers were changed (selected/unselected) during a session in the USFM and
// Filtering tab of preferences.
wxString tempMarkersChangedToBeFiltered;
wxString tempMarkersChangedToBeUnfiltered;
// whm 29Feb2024 added the following flags to determine the filtered state of
// the \x and \xt markers BEFORE any filtering change made by the user during
// this session of the Filter page.
bool marker_X_wasFilteredBeforeThisChange;
bool marker_XT_wasFilteredBeforeThisChange;

wxArrayInt m_userCanSetFilterFlagsDoc; // these flag the userCanSetFilter markers
wxArrayInt* pUserCanSetFilterFlagsDoc; // initialized to point to above in constructor

Expand Down
65 changes: 58 additions & 7 deletions source/helpers.cpp
View file
Original file line number Diff line number Diff line change
Expand Up @@ -3662,6 +3662,10 @@ wxString RemoveMultipleSpaces(wxString& rString)
//return destString;
}

// whm 29Feb2024 modified.
// The modification: We don't wan't the \xt marker to be removed from the filterMkrStr when
// RemoveFilterMarkerFromString() is called with \x as the wholeMarker passed in. We only want
// \xt removed from the filterMkrStr when \xt itself is passed in as wholeMarker to be removed.
void RemoveFilterMarkerFromString(wxString& filterMkrStr, wxString wholeMarker)
{
// if the wholeMarker already exists in filterMkrStr, remove it.
Expand All @@ -3673,6 +3677,10 @@ void RemoveFilterMarkerFromString(wxString& filterMkrStr, wxString wholeMarker)
wholeMarker += _T(' ');
// whm modified 8Jul12. If wholeMarker is \x , \f , or \fe remove it and any associated
// content markers.
// whm 28Feb2024 modification. Since we now show the \xt ...\xt* (stand alone) marker as a
// separate entry within the filter lists in CUSFMFilterPageCommon, we should NOT try to
// remove the \xt marker from the filterMkrStr when wholeMarker is "\\x " is being removed
// from the filterMkrStr list at unfiltering of \x.
if (wholeMarker == _T("\\x "))
{
// Remove the \x marker as well as its associated content markers: \xo \xk \xq \xt \xot \xnt \xdc
Expand All @@ -3686,12 +3694,23 @@ void RemoveFilterMarkerFromString(wxString& filterMkrStr, wxString wholeMarker)
for (i = 0; i < nTot; i++)
{
marker = gpApp->m_crossRefMarkerSet.Item(i);

posn = filterMkrStr.Find(marker);
wxASSERT(posn != wxNOT_FOUND);
if (posn != wxNOT_FOUND)
//wxASSERT(posn != wxNOT_FOUND); // asserting here no longer appropriate
// whm 29Feb2024 added the marker != _T("\\xt ") to the test below, since \xt will
// now only be present when the stand-alone version of \xt is filtered. When \xt is
// not filtered \xt should not be present in filterMkrStr even when \x is filtered
// (see the case when wholeMarker is _T("\\xt ") in the added else if block below).
if (marker != _T("\\xt ") && posn != wxNOT_FOUND)
filterMkrStr.Remove(posn, marker.Length());
}
}
else if (wholeMarker == _T("\\xt "))
{
int posn = filterMkrStr.Find(wholeMarker);
if (posn != wxNOT_FOUND)
filterMkrStr.Remove(posn, wholeMarker.Length());
}
else if (wholeMarker == _T("\\f ") || wholeMarker == _T("\\fe "))
{
// Remove the \f and \fe markers as well as their associated content markers: \fr \fk \fq \fqa \fl
Expand Down Expand Up @@ -3728,6 +3747,15 @@ void RemoveFilterMarkerFromString(wxString& filterMkrStr, wxString wholeMarker)
}
}

// whm 29Feb2024 correction and modified.
// The correction: The array total nTot was gotten from m_crossRefMarkerSet, but it used that
// nTot to access the m_footnoteMarkerSet array. Ouch!! Fortunately there were more elements in
// the m_footnoteMarkerSet (12) than in the m_crossRefMarkerSet (8) so no out-of-bounds error occurs,
// but it couldn't really set the proper cross-ref markers in filterMkrStr; it just attempted to put
// a subset of footnote markers into filterMkrStr.
// The modification: We don't wan't the \xt marker to be added to the filterMkrStr when
// AddFilterMarkerToString() is called with \x as the wholeMarker passed in. We only want
// \xt added to the filterMkrStr when \xt itself is passed in as wholeMarker.
void AddFilterMarkerToString(wxString& filterMkrStr, wxString wholeMarker)
{
// if the wholeMarker does not already exist in filterMkrStr, append it.
Expand All @@ -3740,8 +3768,8 @@ void AddFilterMarkerToString(wxString& filterMkrStr, wxString wholeMarker)

if (wholeMarker == _T("\\x "))
{
// Add the \x marker as well as its associated content markers: \xo \xk \xq \xt \xot \xnt \xdc
// to the filterMkrStr.
// Add the \x marker as well as its associated content markers: \xo \xk \xq \xot \xnt \xdc,
// but NOT \xt to the filterMkrStr.
// Use the wxArrayString m_crossRefMarkerSet which contains the cross reference marker
// plus all of the associated content markers; each includes the initial backslash and following
// space.
Expand All @@ -3750,20 +3778,42 @@ void AddFilterMarkerToString(wxString& filterMkrStr, wxString wholeMarker)
wxString marker;
for (i = 0; i < nTot; i++)
{
marker = gpApp->m_footnoteMarkerSet.Item(i);
if (filterMkrStr.Find(marker) == wxNOT_FOUND)
marker = gpApp->m_crossRefMarkerSet.Item(i); // marker = gpApp->m_footnoteMarkerSet.Item(i); // whm 29Feb2024 correction here!
// whm 29Feb2024 added marker != _T("\\xt") to the following test. We only want \xt added
// to the filterMkrStr when \xt itself is passed in as wholeMarker (see the added else if
// block below where that is taken care of).
if (marker != _T("\\xt ") && filterMkrStr.Find(marker) == wxNOT_FOUND)
{
// The wholeMarker doesn't already exist in the string, so append it.
// NOTE: By appending a marker to the filter marker string we are creating a
// string that can no longer be compared with other filter marker strings by
// means of the == or != operators. Comparison of such filter marker strings will now
// necessarily require a special function StringsContainSameMarkers() be used
// in every place where marker strings are compared.
// whm comment on above NOTE: We could sort the marker elements of the string
// after the addition of marker to make string comparisons easier, but it is
// probably not worth it.
filterMkrStr += marker;
}
}

}
else if (wholeMarker == _T("\\xt ")) // whm 29Feb2024 added this else if block to handle \xt
{
if (filterMkrStr.Find(wholeMarker) == wxNOT_FOUND)
{
// The wholeMarker \xt doesn't already exist in the string, so append it.
// NOTE: By appending a marker to the filter marker string we are creating a
// string that can no longer be compared with other filter marker strings by
// means of the == or != operators. Comparison of such filter marker strings will now
// necessarily require a special function StringsContainSameMarkers() be used
// in every place where marker strings are compared.
// whm comment on above NOTE: We could sort the marker elements of the string
// after the addition of marker to make string comparisons easier, but it is
// probably not worth it.
filterMkrStr += marker;
}
}
else if (wholeMarker == _T("\\f ") || wholeMarker == _T("\\fe "))
{
// Add the \f and \fe markers as well as their associated content markers: \fr \fk \fq \fqa \fl
Expand Down Expand Up @@ -5992,7 +6042,8 @@ wxString FromMergerMakeGstr(CSourcePhrase* pMergedSrcPhrase)
// override of LookupSFM() requires an argument which is a bare marker which is not
// the case here.
USFMAnalysis* pSfm = pDoc->LookupSFM(wxStringBuffer(mkr,mkr.Length())); //pDoc->LookupSFM((wxChar*)mkr.GetData());
if (mkr == _T("\\fig"))
// whm 23Feb2024 added second test || mkr == _T("\\jmp"). If \fig is to be excluded \jmp should be too.
if (mkr == _T("\\fig") || mkr == _T("\\jmp")) // if (mkr == _T("\\fig"))
{
// exclude this one from acceptance
;
Expand Down
Loading

0 comments on commit a776754

Please sign in to comment.