Add dir_entry::refresh and file type observers. Use them in recursive dir_it.

This commit changes behavior of directory_entry constructors and modifiers
that change the stored path in v4: the methods will now automatically query
the filesystem for the file status instead of leaving the cached data
default-initialized. This means that the paths passed to directory_entry
must be valid, otherwise an error will be returned. Filesystem querying
is implemented in the new directory_entry::refresh methods.

The constructors and modifiers that accepted file_status arguments are
now removed in v4. The cached file statuses are an implementation detail,
and eventually we may want to add more cached data, as we add more observers
to directory_entry.

Also added a few file type observers to directory_entry. These observers
allow to avoid querying the filesystem if the full file status is not cached
but the file type is (i.e. when permissions are not cached). This is the case
with readdir-based implementation of directory_iterator, if the underlying
C library supports dirent::d_type field.

recursive_directory_iterator has been updated to use the added file type
observers instead of querying the full status. This may improve performance
of directory iteration.

Closes #288.

Author: Lastique

doc/reference.html

@@ -2543,23 +2543,48 @@ <h2><a name="Class-directory_entry">Class <code>directory_entry</code></a> [clas
         // <a href="#directory_entry-constructors">constructors</a> and destructor
         directory_entry();
         directory_entry(const directory_entry&amp;);
-        explicit directory_entry(const path&amp; p, file_status st=file_status(),
+        explicit directory_entry(const path&amp; p);
+        directory_entry(const path&amp; p, system::error_code&amp; ec);       // v4-only
+        directory_entry(const path&amp; p, file_status st,                // v3-only
           file_status symlink_st=file_status());
        ~directory_entry();
 
         // <a href="#directory_entry-modifiers">modifiers</a>
         directory_entry&amp; operator=(const directory_entry&amp;);
-        void assign(const path&amp; p, file_status st=file_status(),
+        void assign(const path&amp; p);
+        void assign(const path&amp; p, system::error_code&amp; ec);           // v4-only
+        void assign(const path&amp; p, file_status st,                    // v3-only
           file_status symlink_st=file_status());
-        void replace_filename(const path&amp; p, file_status st=file_status(),
+        void replace_filename(const path&amp; p);
+        void replace_filename(const path&amp; p, system::error_code&amp; ec); // v4-only
+        void replace_filename(const path&amp; p, file_status st,          // v3-only
           file_status symlink_st=file_status());
 
+        void refresh();
+        void refresh(system::error_code&amp; ec);
+
         // <a href="#directory_entry-observers">observers</a>
         const path&amp;  path() const;
+
         file_status  status() const;
         file_status  status(system::error_code&amp; ec) const;
         file_status  symlink_status() const;
         file_status  symlink_status(system::error_code&amp; ec) const;
+        file_type file_type() const;
+        file_type file_type(system::error_code&amp; ec) const;
+        file_type symlink_file_type() const;
+        file_type symlink_file_type(system::error_code&amp; ec) const;
+
+        bool exists() const;
+        bool exists(system::error_code&amp; ec) const;
+        bool is_regular_file() const;
+        bool is_regular_file(system::error_code&amp; ec) const;
+        bool is_directory() const;
+        bool is_directory(system::error_code&amp; ec) const;
+        bool is_symlink() const;
+        bool is_symlink(system::error_code&amp; ec) const;
+        bool is_other() const;
+        bool is_other(system::error_code&amp; ec) const;
 
         bool operator&lt; (const directory_entry&amp; rhs);
         bool operator==(const directory_entry&amp; rhs);
@@ -2577,10 +2602,12 @@ <h2><a name="Class-directory_entry">Class <code>directory_entry</code></a> [clas
   }  // namespace filesystem
 }  // namespace boost</pre>
 
-<p>A <code>directory_entry</code> object stores a <code>path object</code>,
-a <code>file_status</code> object for non-symbolic link status, and a <code>file_status</code> object for symbolic link status. The <code>file_status</code> objects act as value caches.</p>
+<p>A <code>directory_entry</code> object stores a <code>path</code> object,
+as well as some amount of cached information about the file identified by the path.
+Currently, the cached information includes a <code>file_status</code> object for non-symbolic
+link status and a <code>file_status</code> object for symbolic link status.</p>
 <blockquote>
-<p>[<i>Note:</i> Because <code>status()</code>on a pathname may be a relatively expensive operation,
+<p>[<i>Note:</i> Because <code>status()</code> on a pathname may be a relatively expensive operation,
 some operating systems provide status information as a byproduct of directory
 iteration. Caching such status information can result is significant time savings. Cached and
 non-cached results may differ in the presence of file system races. <i>—end note</i>]</p>
@@ -2589,8 +2616,14 @@ <h2><a name="Class-directory_entry">Class <code>directory_entry</code></a> [clas
 versus one second for cached status queries. Windows XP, 3.0 GHz processor, with
 a moderately fast hard-drive. Similar speedups are expected on Linux and BSD-derived
 systems that provide status as a by-product of directory iteration.</i></span></p>
-</blockquote>
-<h3> <a name="directory_entry-constructors"> <code>directory_entry </code>constructors</a>
+<p>[<i>Note:</i> The exact set of cached information may vary from one Boost.Filesystem version
+to another, and also between different operating systems and underlying file systems. Users' code
+must not rely on whether a certain piece of information is cached or not. This means that calling
+most observers and modifiers of <code>directory_entry</code> may or may not result in a filesystem
+query that may potentially fail. Information caching is exclusively a performance feature aimed
+at reducing the amount of such queries. <i>—end note</i>]</p>
+</blockquote>
+<h3> <a name="directory_entry-constructors"> <code>directory_entry</code> constructors</a>
 [directory_entry.cons]</h3>
 <pre>directory_entry();</pre>
 <blockquote>
@@ -2614,9 +2647,18 @@ <h3> <a name="directory_entry-constructors"> <code>directory_entry </code>constr
     </tr>
   </table>
 </blockquote>
-<pre>explicit directory_entry(const path&amp; p, file_status st=file_status(), file_status symlink_st=file_status());</pre>
+<pre>explicit directory_entry(const path&amp; p);
+directory_entry(const path&amp; p, system::error_code&amp; ec); // v4-only</pre>
 <blockquote>
-  <p><i>Postcondition:</i></p>
+  <p><i>Effects:</i></p>
+  <p><b>v3:</b> Initializes <code>m_path</code> from <code>p</code> and default-constructs <code>m_status</code> and <code>m_symlink_status</code>.</p>
+  <p>[<i>Note:</i> The cached file statuses will be updated when queried by the caller or by an explicit call to <code>refresh</code>. <i>—end note</i>]</p>
+  <p><b>v4:</b> Initializes <code>m_path</code> from <code>p</code> and calls <code>refresh()</code> or <code>refresh(ec)</code>, respectively.</p>
+  <p><i>Postcondition:</i> <code>path() == p</code> if no error occurs, otherwise <code>path().empty() == true</code>.</p>
+</blockquote>
+<pre>directory_entry(const path&amp; p, file_status st, file_status symlink_st=file_status()); // v3-only</pre>
+<blockquote>
+  <p><b>v3:</b> <i>Postcondition:</i></p>
   <table border="1" cellpadding="5" cellspacing="0" style="border-collapse: collapse" bordercolor="#111111" width="36%">
     <tr>
       <td width="18%"><b>Expression</b></td>
@@ -2636,11 +2678,20 @@ <h3> <a name="directory_entry-constructors"> <code>directory_entry </code>constr
     </tr>
   </table>
 </blockquote>
-<h3> <a name="directory_entry-modifiers"> <code>directory_entry </code>modifiers</a>
+<h3> <a name="directory_entry-modifiers"> <code>directory_entry</code> modifiers</a>
 [directory_entry.mods]</h3>
-<pre>void assign(const path&amp; p, file_status st=file_status(), file_status symlink_st=file_status());</pre>
+<pre>void assign(const path&amp; p);
+void assign(const path&amp; p, system::error_code&amp; ec); // v4-only</pre>
 <blockquote>
-  <p><i>Postcondition:</i></p>
+  <p><i>Effects:</i></p>
+  <p><b>v3:</b> Assigns <code>p</code> to <code>m_path</code> and <code>file_status()</code> to <code>m_status</code> and <code>m_symlink_status</code>.</p>
+  <p>[<i>Note:</i> The cached file statuses will be updated when queried by the caller or by an explicit call to <code>refresh</code>. <i>—end note</i>]</p>
+  <p><b>v4:</b> Assigns <code>p</code> to <code>m_path</code> and calls <code>refresh()</code> or <code>refresh(ec)</code>, respectively. If an error
+  occurs, the value of the cached data is unspecified.</p>
+</blockquote>
+<pre>void assign(const path&amp; p, file_status st, file_status symlink_st=file_status()); // v3-only</pre>
+<blockquote>
+  <p><b>v3:</b> <i>Postcondition:</i></p>
   <table border="1" cellpadding="5" cellspacing="0" style="border-collapse: collapse" bordercolor="#111111" width="36%">
     <tr>
       <td width="18%"><b>Expression</b></td>
@@ -2660,27 +2711,25 @@ <h3> <a name="directory_entry-modifiers"> <code>directory_entry </code>modifiers
     </tr>
   </table>
 </blockquote>
-<pre>void replace_filename(const path&amp; p, file_status st=file_status(), file_status symlink_st=file_status());</pre>
+<pre>void replace_filename(const path&amp; p);
+void replace_filename(const path&amp; p, system::error_code&amp; ec); // v4-only</pre>
 <blockquote>
-  <p><i>Postcondition:</i></p>
-  <table border="1" cellpadding="5" cellspacing="0" style="border-collapse: collapse" bordercolor="#111111" width="43%">
-    <tr>
-      <td width="18%"><b>Expression</b></td>
-      <td width="82%"><b>Value</b></td>
-    </tr>
-    <tr>
-      <td width="18%"><code>path()</code></td>
-      <td width="82%"><code>path().branch() / s</code></td>
-    </tr>
-    <tr>
-      <td width="18%"><code>status()</code></td>
-      <td width="82%"><code>st</code></td>
-    </tr>
-    <tr>
-      <td width="18%"><code>symlink_status()</code></td>
-      <td width="82%"><code>symlink_st</code></td>
-    </tr>
-  </table>
+  <p><i>Effects:</i></p>
+  <p><b>v3:</b> Calls <code>m_path.replace_filename(p)</code> and assigns <code>file_status()</code> to <code>m_status</code> and <code>m_symlink_status</code>.</p>
+  <p>[<i>Note:</i> The cached file statuses will be updated when queried by the caller or by an explicit call to <code>refresh</code>. <i>—end note</i>]</p>
+  <p><b>v4:</b> Calls <code>m_path.replace_filename(p)</code> and then <code>refresh()</code> or <code>refresh(ec)</code>, respectively. If an error
+  occurs, the value of the cached data is unspecified.</p>
+</blockquote>
+<pre>void replace_filename(const path&amp; p, file_status st, file_status symlink_st=file_status()); // v3-only</pre>
+<blockquote>
+  <p><i>Effects:</i> <b>v3:</b> Calls <code>m_path.replace_filename(p)</code> and assigns <code>st</code> to <code>m_status</code> and <code>symlink_st</code>
+  to <code>m_symlink_status</code>.</p>
+</blockquote>
+<pre>void refresh();
+void refresh(system::error_code&amp; ec);</pre>
+<blockquote>
+  <p><i>Effects:</i> Updates any cached data by querying the filesystem about the file identified by <code>m_path</code>. If an error occurs,
+  the value of the cached data is unspecified.</p>
 </blockquote>
 <h3> <a name="directory_entry-observers"> <code>directory_entry</code> observers</a>
 [directory_entry.obs]</h3>
@@ -2691,35 +2740,55 @@ <h3> <a name="directory_entry-observers"> <code>directory_entry</code> observers
 <pre>file_status status() const;
 file_status status(system::error_code&amp; ec) const;</pre>
 <blockquote>
-<p><i>Effects:</i> As if,</p>
-  <blockquote>
-    <pre>if ( !status_known( m_status ) )
-{
-  if ( status_known(m_symlink_status) &amp;&amp; !is_symlink(m_symlink_status) )
-    { m_status = m_symlink_status; }
-  else { m_status = status(m_path<i>[, ec]</i>); }
-}</pre>
-  </blockquote>
+  <p><i>Effects:</i> If <code>!status_known(m_status)</code>, calls <code>refresh()</code> or <code>refresh(ec)</code>, respectively.</p>
   <p><i>Returns:</i> <code>m_status</code></p>
-
   <p><i>Throws:</i> As specified in <a href="#Error-reporting">Error reporting</a>.</p>
-
 </blockquote>
 <pre>file_status  symlink_status() const;
 file_status  symlink_status(system::error_code&amp; ec) const;</pre>
 <blockquote>
-<p>
-  <i>Effects:</i> As if,</p>
-  <blockquote>
-    <pre>if ( !status_known( m_symlink_status ) )
-{
-  m_symlink_status = symlink_status(m_path<i>[, ec]</i>);
-}</pre>
-  </blockquote>
+  <p><i>Effects:</i> If <code>!status_known(m_symlink_status)</code>, calls <code>refresh()</code> or <code>refresh(ec)</code>, respectively.</p>
   <p><i>Returns:</i> <code>m_symlink_status</code></p>
-
   <p><i>Throws:</i> As specified in <a href="#Error-reporting">Error reporting</a>.</p>
-
+</blockquote>
+<pre>file_type file_type() const;
+file_type file_type(system::error_code&amp; ec) const;</pre>
+<blockquote>
+  <p><i>Effects:</i> Equivalent to <code>status().type()</code> or <code>status(ec).type()</code>, respectively.</p>
+  <p>[<i>Note:</i> The implementation may be more efficient than calling <code>status</code>, if the information
+  about the file type is cached, but permissions are not. <i>—end note</i>]</p>
+</blockquote>
+<pre>file_type symlink_file_type() const;
+file_type symlink_file_type(system::error_code&amp; ec) const;</pre>
+<blockquote>
+  <p><i>Effects:</i> Equivalent to <code>symlink_status().type()</code> or <code>symlink_status(ec).type()</code>, respectively.</p>
+  <p>[<i>Note:</i> The implementation may be more efficient than calling <code>symlink_status</code>, if the information
+  about the file type is cached, but permissions are not. <i>—end note</i>]</p>
+</blockquote>
+<pre>bool exists() const;
+bool exists(system::error_code&amp; ec) const;</pre>
+<blockquote>
+  <p><i>Effects:</i> Equivalent to <code>exists(status())</code> or <code>exists(status(ec))</code>, respectively.</p>
+</blockquote>
+<pre>bool is_regular_file() const;
+bool is_regular_file(system::error_code&amp; ec) const;</pre>
+<blockquote>
+  <p><i>Effects:</i> Equivalent to <code>is_regular_file(status())</code> or <code>is_regular_file(status(ec))</code>, respectively.</p>
+</blockquote>
+<pre>bool is_directory() const;
+bool is_directory(system::error_code&amp; ec) const;</pre>
+<blockquote>
+  <p><i>Effects:</i> Equivalent to <code>is_directory(status())</code> or <code>is_directory(status(ec))</code>, respectively.</p>
+</blockquote>
+<pre>bool is_symlink() const;
+bool is_symlink(system::error_code&amp; ec) const;</pre>
+<blockquote>
+  <p><i>Effects:</i> Equivalent to <code>is_symlink(symlink_status())</code> or <code>is_symlink(symlink_status(ec))</code>, respectively.</p>
+</blockquote>
+<pre>bool is_other() const;
+bool is_other(system::error_code&amp; ec) const;</pre>
+<blockquote>
+  <p><i>Effects:</i> Equivalent to <code>is_other(status())</code> or <code>is_other(status(ec))</code>, respectively.</p>
 </blockquote>
 <pre>bool operator==(const directory_entry&amp; rhs);</pre>
 <blockquote>

doc/release_history.html

@@ -41,6 +41,11 @@
 
 <h2>1.83.0</h2>
 <ul>
+  <li>Added <code>directory_entry::refresh</code> method that updates internal cached file statuses for the directory entry identified by path.</li>
+  <li><b>v4:</b> <code>directory_entry</code> constructors and modifiers that initialize or modify the path now automatically call <code>refresh</code>. This may result in errors that were not indicated before and in <b>v3</b>, if querying the filesystem for file statuses fails (e.g. if the file does not exist). This new behavior is similar to std::filesystem.</li>
+  <li><b>v4:</b> <code>directory_entry</code> constructors and methods taking <code>file_status</code> parameters are removed. Users are recommended to remove these arguments and rely on <code>directory_entry</code> calling <code>refresh</code> internally.</li>
+  <li>Added <code>directory_entry</code> member methods for checking the file type of the file, similar to std::filesystem.</li>
+  <li><code>recursive_directory_iterator</code> is now more likely to reuse information about the file type that is obtained during filesystem iteration. This may improve performance. (<a href="https://github.com/boostorg/filesystem/issues/288">#288</a>)</li>
   <li>File streams defined in <code>boost/filesystem/fstream.hpp</code> are now movable, if the standard library file streams are. (<a href="https://github.com/boostorg/filesystem/issues/280">#280</a>)</li>
   <li>Generic <code>path</code> comparison operators are now more restricted to avoid potential ambiguities when user's code contains a <code>using namespace boost::filesystem;</code> directive. (<a href="https://github.com/boostorg/filesystem/issues/285">#285</a>)</li>
   <li>Fixed potential overload resolution ambiguity in users' code, where <code>path</code> constructors from iterators could interfere with function overloads taking a <code>std::initializer_list</code> argument. (<a href="https://github.com/boostorg/filesystem/issues/287">#287</a>)</li>

doc/v4.html

@@ -54,10 +54,13 @@ <h2>Breaking changes</h2>
   <li><a href="reference.html#path-appends"><code>path</code> appends</a> consider root name and root directory of the appended path. If the appended path is absolute, or root name is present and differs from the source path, the resulting path is equivalent to the appended path. If root directory is present, the result is the root directory and relative path rebased on top of the root name of the source path. Otherwise, the behavior is similar to v3. This behavior is similar to C++17 std::filesystem.</li>
   <li><code>path</code> no longer supports construction, assignment or appending from containers of characters. Use string types or iterators as the source for these opereations instead.</li>
   <li><a href="reference.html#path-remove_filename"><code>path::remove_filename</code></a> preserves the trailing directory separator, so that <code>path::has_filename</code> returns <code>false</code> after a successful call to <code>path::remove_filename</code>.</li>
+  <li><code>directory_entry</code> constructors and modifiers that initialize or modify path of the directory entry automatically call <code>directory_entry::refresh</code> instead of clearing cached file statuses. This means that the file identified by the new path needs to be accessible in the filesystem at the point of the call.</li>
+  <li><code>directory_entry</code> constructors and modifiers that accept <code>file_status</code> arguments to initialize cached file statuses are removed. The amount of cached data
+  is an implementation detail of <code>directory_entry</code>, and in the future it may not be limited to just file statuses. Users should rely on automatic refreshes of the cached data.</li>
 </ul>
 
 <hr>
-<p>&copy; Copyright Andrey Semashev, 2021-2022</p>
+<p>&copy; Copyright Andrey Semashev, 2021-2023</p>
 <p>Distributed under the Boost Software License, Version 1.0. See
 <a href="http://www.boost.org/LICENSE_1_0.txt">www.boost.org/LICENSE_1_0.txt</a></p>