blog post image
Andrew Lock avatar

Andrew Lock

~12 min read

Syncing a git branch between Windows and WSL filesystems

In this post I show how you can work on a git branch with a repository checked out in Windows, and then sync that branch to another git folder in WSL, without having to push to a remote server like GitHub. I start by discussing why you might want to do that, and then show how to make it work!

Why would you want to do that?

This post was driven from a colleague asking me how to do the following (paraphrased):

I have a git repository checked out in Windows that I'm working on in Visual Studio. I want to work on the branch in Windows and then sync it to Windows Subsystem for Linux (WSL) to build and test the changes.

Now, you might be thinking that seems a bit unnecessary. After all, your Windows drive is mounted inside WSL (at /mnt/c), so it would be perfectly possible to simply access the folder from both sides. Why not use your IDE to edit the files in Windows at C:/repos/my-repo, and run it in WSL from /mnt/c/repos/my-repo?

The main reason you might not want to do this is performance. Basically it's really slow to access the file system when you "cross the streams" and have Windows and Linux accessing the same files. Essentially the file systems are "remote" file systems, with every IO operation requiring a "network" hop over a Hyper-V socket. That makes it really quite slow.

In fact, this is one of the few cases where WSL 1 is "better" than WSL 2; it's a lot faster to access the Windows file system, something like 10× faster. You'd still be hard-pressed to recommend using WSL 1 these days though!

So to avoid hitting these issues, the recommendation is to only access the Windows file system from your Windows applications, and only access the WSL file system from your WSL applications. Which means you need two different folders if you're in the situation my colleague was describing.

VS Code has a "remote" extension that lets you edit you easily edit your WSL files by using a client-server architecture. Visual Studio and Rider both also have similar mechanisms, but frankly I've not had much success with either of them, so I tend to take a similar approach to my colleague!

So that's the setup. I want to be able to edit files in my IDE on Windows, commit the changes, and then sync the folder to the WSL copy, and run my tests on the Linux side. As a bonus, it would be nice to be able to do the reverse too, syncing any edits in the WSL copy back to the Windows side.

Luckily, that functionality is all built into git natively!

Centralised or distributed git?

Somewhat ironically, when people think about using git these days they normally think of a centralised "remote" service, such as GitHub or GitLab, as the source of truth for a git repository. You make your changes locally, and you push them to the remote git server.

This centralised approach has some advantages but fundamentally git is designed to be used in a distributed fashion, where each instance of the repository is equal in terms of status. There's no central authority; everyone has slightly different versions of the data, and they can be synced between each other.

These other instances of the repository are called remotes. If you run git remote -v in a repository, you're probably used to seeing something like this:

> git remote -v
origin  https://github.com/DataDog/dd-trace-dotnet.git (fetch)
origin  https://github.com/DataDog/dd-trace-dotnet.git (push)

In this example, the remote is a GitHub repository. The configuration shows where your commits should be pushed to and pulled from. But there's nothing stopping you setting an additional remote that points to a different instance of the repository on your file system. This is the "trick" we're going to use to sync between two different folders.

It's also worth noting that you can have different values for the fetch and push URLs, though that isn't very common. You can potentially set these to two different URLs to support "triangular workflows", where you push to a fork, open a Pull Request to the main repository, and then fetch form the main repository. Personally I would still treat these as two separate remotes.

So the simple approach to syncing between folders on the same machine is to set them up as remotes. In the next section I'll show how to do that.

Cloning a git repository from a Windows folder into a WSL folder

For this example we'll assume you have a git repository in a folder on Windows, in which you're working on a fix. In this example, we're updating a project to support .NET 8. You start a branch called update-to-dotnet-8 and commit your changes. You're already syncing your changes to GitHub, so you already have a remote called origin

The output from gitk for the current branch

Now let's head over to the WSL side. We want to test the update-to-dotnet-8 branch on Linux, so we clone the repository, using the "WSL" path to the git repository in Windows (i.e. the /mnt/c path) as the remote. I also changed the name of the remote to windows instead of origin, to make it more obvious what's going on.

cd ~       # move to the home folder in WSL, in the WSL file-system
git clone -o windows /mnt/c/repos/my-test/       # clone the repo from Windows into WSL
cd ./my-test       # move into the cloned directory

We can use git log to check what the git repository looks like on the WSL side, using some extra options to prettify it:

$ git log --all --decorate --oneline --graph
* 79bac8e (HEAD -> update-to-dotnet-8, windows/update-to-dotnet-8, windows/HEAD) Updating to .NET 8
* e72a87b (windows/master) Update to net5.0
* 88079cf Simplify ProjectReference to analyzer
* 04ef379 Simplified it even further...
* 2c077ea It is Alive!
* dc5b093 Add example
* 7053a8c Initial commit

Which looks prettier and easier to read in the console:

The output from git log --all --decorate --oneline --graph for WSL

So it's as simple as that; we can now build and test our project on the WSL side, and benefit from the fast file system, without having to access the Windows file system.

If we make changes on the Windows side, and commit them:

echo "some change" > some_text.txt
git add .
git commit -m "More changes"

Then we can pull those changes into the WSL repository with a simple git pull:

$ git pull
remote: Enumerating objects: 4, done.
remote: Counting objects: 100% (4/4), done.
remote: Compressing objects: 100% (2/2), done.
remote: Total 3 (delta 1), reused 0 (delta 0)
Unpacking objects: 100% (3/3), 283 bytes | 47.00 KiB/s, done.
From /mnt/c/repos/my-test
   79bac8e..e372c87  update-to-dotnet-8 -> windows/update-to-dotnet-8
Updating 0959391..e372c87
Fast-forward
 some_text.txt | Bin 0 -> 34 bytes
 1 file changed, 0 insertions(+), 0 deletions(-)
 create mode 100644 some_text.txt

But what happens if we make changes on the WSL side and want to push them back to the Windows side?

Pushing changes from WSL to Windows

Let's imagine we've made some changes on the WSL side:

touch somefile.txt                 # Create the file somefile.txt
git add .                          # Stage the file in the index
git commit -m "Made some fixes"    # Commit the changes

We now have an extra commit in update-to-dotnet-8 on the WSL side:

$ git log -a --decorate --oneline --graph
* 0959391 (HEAD -> update-to-dotnet-8) Made some fixes
* 79bac8e (windows/update-to-dotnet-8, windows/HEAD) Updating to .NET 8
* e72a87b (windows/master) Update to net5.0
* 88079cf Simplify ProjectReference to analyzer
* 04ef379 Simplified it even further...
* 2c077ea It is Alive!
* dc5b093 Add example
* 7053a8c Initial commit

The output from git log for WSL after making a commit

So, we should just be able to push that to the Windows side (the windows remote) using git push, right?

$ git push windows
Enumerating objects: 7, done.
Counting objects: 100% (7/7), done.
Delta compression using up to 4 threads
Compressing objects: 100% (4/4), done.
Writing objects: 100% (4/4), 395 bytes | 395.00 KiB/s, done.
Total 4 (delta 2), reused 0 (delta 0)
remote: error: refusing to update checked out branch: refs/heads/update-to-dotnet-8
remote: error: By default, updating the current branch in a non-bare repository
remote: is denied, because it will make the index and work tree inconsistent
remote: with what you pushed, and will require 'git reset --hard' to match
remote: the work tree to HEAD.
remote:
remote: You can set the 'receive.denyCurrentBranch' configuration variable
remote: to 'ignore' or 'warn' in the remote repository to allow pushing into
remote: its current branch; however, this is not recommended unless you
remote: arranged to update its work tree to match what you pushed in some
remote: other way.
remote:
remote: To squelch this message and still keep the default behaviour, set
remote: 'receive.denyCurrentBranch' configuration variable to 'refuse'.
To /mnt/c/repos/my-test/
 ! [remote rejected] update-to-dotnet-8 -> update-to-dotnet-8 (branch is currently checked out)
error: failed to push some refs to '/mnt/c/repos/my-test/'

Oh dear, that's not gone well. Basically, it's not possible to safely push to a branch when it's checked out in another folder. That gives us two options:

  • Check out a different branch on the Windows side before trying to push from the WSL side, and then re-check out the original branch afterwards
  • Configure WSL as a remote on the Windows side and pull the changes.

I'll talk through both options in the following sections.

Switching branches to allow pushing from WSL to Windows

The error message we got when trying to push our changes from WSL to Windows says that it could cause problems because you might have staged and unstaged changes on the Windows side for the branch you're trying to push to. If you "force" push to the branch, you could end up with things in an inconsistent state. In theory, if you don't have any local changes and nothing is staged, then this shouldn't actually be a problem. But given the potential for issues, I wouldn't recommend trying to work around it. Instead we'll take the simple approach of

  • Checkout master on the Windows side
  • Push the changes from WSL to Windows
  • Checkout update-to-dotnet-8 on the Windows side

The first command we run on Windows:

> git checkout master
Switched to branch 'master'
Your branch is up to date with 'origin/master'.

We then switch back to WSL:

$ git push windows
Enumerating objects: 4, done.
Counting objects: 100% (4/4), done.
Delta compression using up to 4 threads
Compressing objects: 100% (2/2), done.
Writing objects: 100% (3/3), 276 bytes | 276.00 KiB/s, done.
Total 3 (delta 1), reused 0 (delta 0)
To /mnt/c/repos/my-test/
   79bac8e..0959391  update-to-dotnet-8 -> update-to-dotnet-8

And finally we switch back to Windows

> git checkout update-to-dotnet-8
Switched to branch 'update-to-dotnet-8'

And we've successfully pushed our commit from WSL to Windows:

The output from gitk for the current branch after pushing a commit from WSL

It's a bit tedious to have to switch back and forth repeatedly, so initially I considered running the git checkout master command from the WSL side, but interestingly that didn't work:

$ git -C /mnt/c/repos/my-test checkout master
error: Your local changes to the following files would be overwritten by checkout:
        ConsoleTest/ConsoleTest.csproj
Please commit your changes or stash them before you switch branches.
Aborting

What's strange here, is that I don't have any local changes. If we run the git status command from the Windows side we get:

> git status
On branch update-to-dotnet-8
nothing to commit, working tree clean

But if we run the same command, on the same Windows folder, from the WSL side:

$ git status
On branch update-to-dotnet-8
Changes not staged for commit:
  (use "git add <file>..." to update what will be committed)
  (use "git restore <file>..." to discard changes in working directory)
        modified:   .gitignore
        modified:   ConsoleTest/ConsoleTest.csproj
        modified:   ConsoleTest/Program.cs
        modified:   Generators/Generators.csproj
        modified:   Generators/HelloWorldGenerator.cs
        modified:   global.json
        modified:   source-generator-test.sln

no changes added to commit (use "git add" and/or "git commit -a")

As you can see, when we run from the WSL side git sees loads of changes! So what's going on here?

My assumption is that this is down to differences in the global git config. I think simple differences in line ending behaviour should explain what we're seeing: git on Windows checks out text files with Windows-style \r\n line endings but commits Linux-style \n; git on Linux uses \n everywhere.

So it looks like you're stuck switching back and forth if you want to push your commits from WSL to Windows. But there's another option: pulling the commits.

Pulling the commits from WSL to Windows

When we created the WSL folder, we used the Windows file system as a remote (called windows) so that we could pull the branches from it. Instead of trying to push commits to it (like you would push commits to GitHub), we can take the opposite approach and configure the WSL git repository as a remote (called wsl) in the Windows git repository. We can then pull the commits from WSL into the Windows repository.

First of all, we have a one-time setup command to run, to add the WSL repository as a remote in the Windows repository. I'm using the full UNC file path to the WSL repository (inside my WSL instance called Ubuntu, and my WSL username of andrewlock). I call this remote wsl (but you can choose anything you like). Run the following from your Windows repository:

git remote add wsl \\wsl$\Ubuntu\home\andrewlock\my-test

If you check the remote configuration for the Windows repository you'll see it now has two remotes:

> git remote -v
origin  https://github.com/andrewlock/source-generator-test.git (fetch)
origin  https://github.com/andrewlock/source-generator-test.git (push)
wsl     \\wsl$\Ubuntu\home\andrewlock\my-test (fetch)
wsl     \\wsl$\Ubuntu\home\andrewlock\my-test (push)

And if you run a git fetch wsl to populate the local repository:

> git fetch wsl
From \\wsl$\Ubuntu\home\andrewlock\my-test
 * [new branch]      update-to-dotnet-8 -> wsl/update-to-dotnet-8

Then you'll see that the remote branch is available as you'd expect:

The output from gitk after running a git fetch wsl

You can pull the changes from the WSL side back into the Windows repository by running git pull wsl update-to-dotnet-8 (or git merge wsl/update-to-dotnet-8):

> git pull wsl update-to-dotnet-8
From \\wsl$\Ubuntu\home\andrewlock\my-test
 * branch            update-to-dotnet-8 -> FETCH_HEAD
Updating 79bac8e..0959391
Fast-forward
 somefile.txt | 0
 1 file changed, 0 insertions(+), 0 deletions(-)
 create mode 100644 somefile.txt

Note that in this case you can't just run git pull wsl without specifying a branch:

> git pull wsl
You asked to pull from the remote 'wsl', but did not specify
a branch. Because this is not the default configured remote
for your current branch, you must specify a branch on the command line.

The first time you push a branch to a remote, git configures that as the default origin for the branch. As we haven't run git push for the update-to-dotnet-8 branch yet, that isn't configured. However, in general you probably don't want to configure the wsl remote as the default for a branch anyway, otherwise you might think you're pushing a branch to your GitHub repo whereas you're actually pushing it to your WSL copy!

The complete flow

For completeness, lets run through the complete flow for creating a WSL copy and syncing changes back and forth with a Windows repository.

  1. Clone the Windows repository in WSL: git clone -o windows /mnt/c/path-to-your-repo/
  2. Configure the WSL remote in Windows: git remote add wsl \\wsl$\Ubuntu\home\username\path-to-repo
  3. Pull changes into WSL from Windows: git pull or git pull windows
  4. Pull changes into Windows from WSL: git pull wsl some-branch (note the remote and branch name are typically required in this case)

And that's all there is to it!

Summary

In this post I described how to synchronise two git folders by configuring them as remotes. This is useful, for example, if you want to have a git repository both in Windows and inside the WSL filesystem. Accessing the other file system in this situation is generally slow so is not recommended. By treating each git folder as a remote repository it's easy to git pull your changes in both directions.

Andrew Lock | .Net Escapades
Want an email when
there's new posts?