NTFS reparse point
An NTFS reparse point is a type of NTFS file system object. It is available with the NTFS v3.0 found in Windows 2000 or later versions. Reparse points provide a way to extend the NTFS filesystem. A reparse point contains a reparse tag and data that are interpreted by a filesystem filter identified by the tag. Microsoft includes several default tags including NTFS symbolic links, directory junction points, volume mount points and Unix domain sockets. Also, reparse points are used as placeholders for files moved by Windows 2000's Remote Storage Hierarchical Storage System. They also can act as hard links, but aren't limited to point to files on the same volume: they can point to directories on any local volume. The feature is inherited to ReFS.
The open source NTFS-3G driver implements built-in support for the link-type reparse points, namely symbolic links and junction points. A plugin filter system is available to handle additional types of reparse points, allowing for chunk-deduplicated files, system-compressed files, and OneDrive files to be read.
Structure
A reparse point has the following general structure, in C struct form:struct REPARSE_BUFFER
The reparse tag is unique to each type of reparse point. It defines to which reparse point handler the I/O manager delegates the processing. Microsoft provides documentation on some "public" tag types.
Types
Volume mount points
s are similar to Unix mount points, where the root of another file system is attached to a directory. In NTFS, this allows additional file systems to be mounted without requiring a separate drive letter for each.Once a volume has been mounted on top of an existing directory of another volume, the contents previously listed in that directory become invisible and are replaced by the content of the root directory of the mounted volume. The mounted volume could still have its own drive letter assigned separately. The file system does not allow volumes to be mutually mounted on each other. Volume mount points can be made to be either persistent or not persistent.
Mounted volumes may use other file systems than just NTFS, possibly with their own security settings and remapping of access rights according to the remote file system policy.
The substitute names of volume mount points use the NT namespace form. Junctions generally use to refer to a volume with an existing driver letter, while true volume mount points use to refer to any volume. UNC paths are invalid for junctions.
Directory junctions
are defined using the exact same mechanism as volume mount points are. The only difference is that their substitute names point to a subdirectory of another volume that usually already has a drive letter. This function is conceptually similar to symbolic links to directories in Unix, except that the target in NTFS must always be another directory.For instance, the directory
C:\exampledir with a directory junction attribute that contains a link to D:\linkeddir will automatically refer to the directory D:\linkeddir when it is accessed by a user-mode application.Directory junctions are persistent, and resolved on the server side as they share the same security realm of the local system or domain on which the parent volume is mounted and the same security settings for its contents as the content of the target directory; however the junction itself may have distinct security settings. Unlinking a directory junction does not delete files in the target directory.
Some directory junctions are installed by default on Windows Vista, for compatibility with previous versions of Windows, such as
Documents and Settings in the root directory of the system drive, which links to the Users physical directory in the root directory of the same volume. However they are hidden by default, and their security settings are set up so that the Windows Explorer will refuse to open them from within the Shell or in most applications, except for the local built-in SYSTEM user or the local Administrators group. This additional security restriction has probably been made to avoid users of finding apparent duplicate files in the joined directories and deleting them by error, because the semantics of directory junctions are not the same as for hard links; the reference counting is not used on the target contents and not even on the referenced container itself.Directory junctions are soft links, working as a limited form of symbolic links, but it is an optimized version allowing faster processing of the reparse point with which they are implemented, with less overhead than the newer NTFS symbolic links, and can be resolved on the server side.
Symbolic links
s were introduced in Windows Vista. Symbolic links are resolved on the client side. So when a symbolic link is shared, the target is subject to the access restrictions on the client, and not the server.Symbolic links can be created either to files or to directories, but the semantic of the link must be provided with the created link. The target however need not exist or be available when the symbolic link is created: when the symbolic link will be accessed and the target will be checked for availability, NTFS will also check if it has the correct type ; it will return a not-found error if the existing target has the wrong type.
They can also reference shared directories on remote hosts or files and subdirectories within shared directories: their target is not mounted immediately at boot, but only temporarily on demand while opening them with the
OpenFile or CreateFile API. Their definition is persistent on the NTFS volume where they are created.The symbolic link data is similar to mount point data, in that both use an NT namespace path. The difference is that symbolic links accepts UNC paths, but not Volume mounts.
Distributed Link Tracking (DLT)
Distributed link tracking allows applications to track files, shell shortcuts or OLE links even if they were renamed or moved to another volume within the same machine, domain or workgroup. Tracking is implemented as a system service, which uses the object identifier index stored in a metafile. When the application requests a track to a file or directory, the tracking service creates the OID entry, which points to the file, and file rename, copy or move operation to a NTFS v3 volume also copies the object ID. This allows the tracking service to eventually find the target file.Data deduplication
When there are several directories that have different but similar files, some of these files may have identical content. Single instance storage, found in Windows Server 2000 through Windows Storage Server 2008, allows identical files to be merged to one file and create references to that merged file. SIS consists of a file system filter that manages copies, modification and merges to files; and a user space service that searches for files that are identical and need merging. SIS was mainly designed for remote installation servers as these may have multiple installation images that contain many identical files; SIS allows these to be consolidated but, unlike for example hard links, each file remains distinct; changes to one copy of a file will leave others unaltered. This is similar to copy-on-write, which is a technique by which memory copying is not really done until one copy is modified.Since Windows Server 2012, there is a new chunk-based data deduplication mechanism that allows files with similar content to be deduplicated as long as they have stretches of identical data. This mechanism is more powerful than SIS. Since Windows Server 2019, the feature is fully supported on ReFS.
Hierarchical Storage Management (HSM)
is a means of transferring files that are not used for some period of time to less expensive storage media. When the file is next accessed, the reparse point on that file determines that it is needed and retrieves it from storage.Native Structured Storage (NSS)
NSS was an ActiveX document storage technology that has since been discontinued by Microsoft. It allowed ActiveX Documents to be stored in the same multi-stream format that ActiveX uses internally. An NSS file system filter was loaded and used to process the multiple streams transparently to the application, and when the file was transferred to a non-NTFS formatted disk volume it would also transfer the multiple streams into a single stream.Unix domain socket (socket)
With windows 10 build 17063, Microsoft introduced Unix domain sockets to windows, these are realized by using the afunix.sys kernel driver and a new reparse point in the filesystem. Unix domain sockets are common on BSD and Linux systems since ages and can be seen as the standard for inter process communication on these systems. Therefore their introduction to windows will allow simplified adoption of code and cross platform portability.System compression
introduces a few read-only compression algorithms for NTFS, taken from Windows Imaging Format. They are XPRESS4K/8K/16K and LZX. Both are based on LZ77 with Huffman entropy coding, which LZNT1 lacked. They are mainly used for new CompactOS feature, which compresses the entire system partition using one of these algorithms. They can also be manually turned on per file with the flag of the command. The algorithms divide files by chunks with a fragmentation behavior similar to that of vanilla NTFS compression.Internally, the file is recorded as a reparse point with tag 0x80000017 to record the fact that the file has been specially compressed, and the actual data is stored in an alternate data stream named "WofCompressedData". The new design is meant purely for read-only access, so any writes to compressed files result in fully decompressing the file on Windows.