[release-branch.go1.21] os: report IO_REPARSE_TAG_DEDUP files as regular in Stat...

Prior to CL 460595, Lstat reported most reparse points as regular
files. However, reparse points can in general implement unusual
behaviors (consider IO_REPARSE_TAG_AF_UNIX or IO_REPARSE_TAG_LX_CHR),
and Windows allows arbitrary user-defined reparse points, so in
general we must not assume that an unrecognized reparse tag represents
a regular file; in CL 460595, we began marking them as irregular.

As it turns out, the Data Deduplication service on Windows Server runs
an Optimization job that turns regular files into reparse files with
the tag IO_REPARSE_TAG_DEDUP. Those files still behave more-or-less
like regular files, in that they have well-defined sizes and support
random-access reads and writes, so most programs can treat them as
regular files without difficulty. However, they are still reparse
files: as a result, on servers with the Data Deduplication service
enabled, files could arbitrarily change from “regular” to “irregular”
without explicit user intervention.

Since dedup files are converted in the background and otherwise behave
like regular files, this change adds a special case to report DEDUP
reparse points as regular.

Fixes #63764.
Updates #63429.

No test because to my knowledge we don't have any Windows builders
that have the deduplication service enabled, nor do we have a way to
reliably guarantee the existence of an IO_REPARSE_TAG_DEDUP file.

(In theory we could add a builder with the service enabled on a
specific volume, write a test that encodes knowledge of that volume,
and use the GO_BUILDER_NAME environment variable to run that test only
on the specially-configured builders. However, I don't currently have
the bandwidth to reconfigure the builders in this way, and given the
simplicity of the change I think it is unlikely to regress
accidentally.)

Change-Id: I649e7ef0b67e3939a980339ce7ec6a20b31b23a1
Cq-Include-Trybots: luci.golang.try:go1.21-windows-amd64-longtest
Reviewed-on: https://go-review.googlesource.com/c/go/+/538218
Reviewed-by: Alex Brainman <alex.brainman@gmail.com>
Reviewed-by: David Chase <drchase@google.com>
Auto-Submit: Heschi Kreinick <heschi@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>

diff --git a/src/internal/syscall/windows/reparse_windows.go b/src/internal/syscall/windows/reparse_windows.go
index 6e111392f09169d174f00f9d38c2d62538ca7832..6caf47e8676c9c89b0826c3ff766b348b4a5c1e0 100644 (file)
--- a/src/internal/syscall/windows/reparse_windows.go
+++ b/src/internal/syscall/windows/reparse_windows.go
@@ -12,6 +12,7 @@ import (
 const (
        FSCTL_SET_REPARSE_POINT    = 0x000900A4
        IO_REPARSE_TAG_MOUNT_POINT = 0xA0000003
+       IO_REPARSE_TAG_DEDUP       = 0x80000013
 
        SYMLINK_FLAG_RELATIVE = 1
 )

diff --git a/src/os/types_windows.go b/src/os/types_windows.go
index 9a3d508783940ab64a718abd72b124cee17d8d89..effb0148dfd5edc5d8bb086b533cc5ffe64fa71d 100644 (file)
--- a/src/os/types_windows.go
+++ b/src/os/types_windows.go
@@ -141,7 +141,23 @@ func (fs *fileStat) Mode() (m FileMode) {
                m |= ModeDevice | ModeCharDevice
        }
        if fs.FileAttributes&syscall.FILE_ATTRIBUTE_REPARSE_POINT != 0 && m&ModeType == 0 {
-               m |= ModeIrregular
+               if fs.ReparseTag == windows.IO_REPARSE_TAG_DEDUP {
+                       // If the Data Deduplication service is enabled on Windows Server, its
+                       // Optimization job may convert regular files to IO_REPARSE_TAG_DEDUP
+                       // whenever that job runs.
+                       //
+                       // However, DEDUP reparse points remain similar in most respects to
+                       // regular files: they continue to support random-access reads and writes
+                       // of persistent data, and they shouldn't add unexpected latency or
+                       // unavailability in the way that a network filesystem might.
+                       //
+                       // Go programs may use ModeIrregular to filter out unusual files (such as
+                       // raw device files on Linux, POSIX FIFO special files, and so on), so
+                       // to avoid files changing unpredictably from regular to irregular we will
+                       // consider DEDUP files to be close enough to regular to treat as such.
+               } else {
+                       m |= ModeIrregular
+               }
        }
        return m
 }