os: report IO_REPARSE_TAG_DEDUP files as regular in Stat and Lstat

Prior to CL 460595, Lstat reported most reparse points as regular
files. However, reparse points can in general implement unusual
behaviors (consider IO_REPARSE_TAG_AF_UNIX or IO_REPARSE_TAG_LX_CHR),
and Windows allows arbitrary user-defined reparse points, so in
general we must not assume that an unrecognized reparse tag represents
a regular file; in CL 460595, we began marking them as irregular.

As it turns out, the Data Deduplication service on Windows Server runs
an Optimization job that turns regular files into reparse files with
the tag IO_REPARSE_TAG_DEDUP. Those files still behave more-or-less
like regular files, in that they have well-defined sizes and support
random-access reads and writes, so most programs can treat them as
regular files without difficulty. However, they are still reparse
files: as a result, on servers with the Data Deduplication service
enabled, files could arbitrarily change from “regular” to “irregular”
without explicit user intervention.

Since dedup files are converted in the background and otherwise behave
like regular files, this change adds a special case to report DEDUP
reparse points as regular.

Fixes #63429.

No test because to my knowledge we don't have any Windows builders
that have the deduplication service enabled, nor do we have a way to
reliably guarantee the existence of an IO_REPARSE_TAG_DEDUP file.

(In theory we could add a builder with the service enabled on a
specific volume, write a test that encodes knowledge of that volume,
and use the GO_BUILDER_NAME environment variable to run that test only
on the specially-configured builders. However, I don't currently have
the bandwidth to reconfigure the builders in this way, and given the
simplicity of the change I think it is unlikely to regress
accidentally.)

Change-Id: I649e7ef0b67e3939a980339ce7ec6a20b31b23a1
Cq-Include-Trybots: luci.golang.try:gotip-windows-amd64-longtest
Reviewed-on: https://go-review.googlesource.com/c/go/+/537915
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: David Chase <drchase@google.com>
Reviewed-by: Alex Brainman <alex.brainman@gmail.com>
Reviewed-by: Quim Muntal <quimmuntal@gmail.com>
Auto-Submit: Bryan Mills <bcmills@google.com>

diff --git a/src/internal/syscall/windows/reparse_windows.go b/src/internal/syscall/windows/reparse_windows.go
index a5bc4963c21a9ba637bb6521f2842a62dee9206b..02f32c6752592f0d265724d12f8c8878d9281dd2 100644 (file)
--- a/src/internal/syscall/windows/reparse_windows.go
+++ b/src/internal/syscall/windows/reparse_windows.go
@@ -12,6 +12,7 @@ import (
 const (
        FSCTL_SET_REPARSE_POINT    = 0x000900A4
        IO_REPARSE_TAG_MOUNT_POINT = 0xA0000003
+       IO_REPARSE_TAG_DEDUP       = 0x80000013
 
        SYMLINK_FLAG_RELATIVE = 1
 )

diff --git a/src/os/types_windows.go b/src/os/types_windows.go
index b457410a4fee1884a5f7a5b055765696cf3ef96d..6b9fef6c123f61e125c6200a60b0a59f9a498fc6 100644 (file)
--- a/src/os/types_windows.go
+++ b/src/os/types_windows.go
@@ -185,7 +185,23 @@ func (fs *fileStat) Mode() (m FileMode) {
                m |= ModeDevice | ModeCharDevice
        }
        if fs.FileAttributes&syscall.FILE_ATTRIBUTE_REPARSE_POINT != 0 && m&ModeType == 0 {
-               m |= ModeIrregular
+               if fs.ReparseTag == windows.IO_REPARSE_TAG_DEDUP {
+                       // If the Data Deduplication service is enabled on Windows Server, its
+                       // Optimization job may convert regular files to IO_REPARSE_TAG_DEDUP
+                       // whenever that job runs.
+                       //
+                       // However, DEDUP reparse points remain similar in most respects to
+                       // regular files: they continue to support random-access reads and writes
+                       // of persistent data, and they shouldn't add unexpected latency or
+                       // unavailability in the way that a network filesystem might.
+                       //
+                       // Go programs may use ModeIrregular to filter out unusual files (such as
+                       // raw device files on Linux, POSIX FIFO special files, and so on), so
+                       // to avoid files changing unpredictably from regular to irregular we will
+                       // consider DEDUP files to be close enough to regular to treat as such.
+               } else {
+                       m |= ModeIrregular
+               }
        }
        return m
 }