A silent voice begins to softly speak.

A breath of wind awakens sleeping words.

Reverse Proxy to Prevent Microsoft Clarity from Being Blocked by Ad Blocker Rules

· / ·

reads

AI Translation中文English

AI-generated summary

AI · GEN

Introduction

Microsoft Clarity can analyze user behavior on websites, and this site has also integrated it.
Personally, I find it quite useful. You can check the website's traffic whenever you have free time.
It's also very simple to use - just add the line of code provided by the dashboard into the <head> tag.

However, many ad blocker rules will block Clarity's domain, resulting in missing user connection information. This can be avoided by using nginx reverse proxy to your own domain. Here are the specific steps for your reference.

Analyzing Requests

Below is an example of code added in the <head> tag.

CodeBlock Loading...

Or you can open the browser's developer tools to capture packets and see that the relevant code will first request https://www.clarity.ms/tag/abcdefg , which downloads a JavaScript script:

CodeBlock Loading...

There are 3 links in it:

These links need to be replaced one by one.

Replacing Links and Setting Up Reverse Proxy

Assuming your domain is example.com, you can create a subdomain clarity.example.com and point it to a public nginx server.

At this point, you can save the script requested from https://www.clarity.ms/tag/abcdefg to /var/www/html/tag/abcdefg.

CodeBlock Loading...

Then replace the 3 links above as follows:

After that, you also need to download https://www.clarity.ms/s/0.8.13-beta/clarity.js to /var/www/html/s/0.8.13-beta/clarity.js. The version may be different when you read this, so just modify accordingly. The /var/www/html directory above can also be changed, but pay attention to permissions to ensure the nginx process can access it. The directory structure is as follows:

CodeBlock Loading...

nginx reverse proxy configuration reference:

CodeBlock Loading...

Reconfiguring the Website

Replace www.clarity.ms in the script originally added to your website's <head> with clarity.example.com. That's it! Try visiting your website and check if there are real-time online users in the dashboard!

Drawbacks

The drawback after doing this is that the dashboard will show all visits as coming from the nginx server's location. It would be great if Microsoft could determine geographic location from request headers.
It cannot automatically follow Microsoft's clarity.js upgrades, but you can write a script to automate it. Reference code:

CodeBlock Loading...

Introduction

Analyzing Requests

Below is an example of code added in the <head> tag.

HTML

<script type="text/javascript">
    (function(c,l,a,r,i,t,y){
        c[a]=c[a]||function(){(c[a].q=c[a].q||[]).push(arguments)};
        t=l.createElement(r);t.async=1;t.src="https://www.clarity.ms/tag/"+i;
        y=l.getElementsByTagName(r)[0];y.parentNode.insertBefore(t,y);
    })(window, document, "clarity", "script", "abcdefg");
</script>

CodeBlock Loading...

Or you can open the browser's developer tools to capture packets and see that the relevant code will first request https://www.clarity.ms/tag/abcdefg , which downloads a JavaScript script:

(function (c, l, a, r, i, t, y) {
    function sync() {
        (new Image()).src = "https://c.clarity.ms/c.gif";
    }

    if ("complete" == document.readyState) {
        sync();
    } else {
        window.addEventListener("load", sync);
    }

    if (a[c].v || a[c].t) {
        return a[c]("event", c, "dup." + i.projectId);
    }

    a[c].t = true;
    t = l.createElement(r);
    t.async = true;
    t.src = "https://www.clarity.ms/s/0.8.13-beta/clarity.js";

    y = l.getElementsByTagName(r)[0];
    y.parentNode.insertBefore(t, y);

    a[c]("start", i);
    a[c].q.unshift(a[c].q.pop());
    a[c]("set", "C_IS", "0");
})(
    "clarity",
    document,
    window,
    "script",
    {
        projectId: "abcdefg",
        upload: "https://z.clarity.ms/collect",
        expire: 365,
        cookies: ["_uetmsclkid", "_uetvid"],
        track: true,
        content: true,
        unmask: ["body"],
        dob: 2002
    }
);

CodeBlock Loading...

There are 3 links in it:

These links need to be replaced one by one.

Replacing Links and Setting Up Reverse Proxy

Assuming your domain is example.com, you can create a subdomain clarity.example.com and point it to a public nginx server.

At this point, you can save the script requested from https://www.clarity.ms/tag/abcdefg to /var/www/html/tag/abcdefg.

BASH

mkdir -p /var/www/html/tag
wget https://www.clarity.ms/tag/abcdefg -O /var/www/html/tag/abcdefg

CodeBlock Loading...

Then replace the 3 links above as follows:

> tree /var/www/html
/var/www/html
├── clarity.js
├── s
│   ├── 0.8.13-beta
│   │   └── clarity.js
│   └── 0.8.9
│       └── clarity.js
└── tag
    ├── abcdefg
    └── hijklmn

5 directories, 5 files

CodeBlock Loading...

nginx reverse proxy configuration reference:

NGINX

server {
    listen 80;
    listen [::]:80;
    listen 443 ssl;
    listen [::]:443 ssl;
    server_name clarity.example.com;

    if ($scheme = http) {
        return 301 https://$host$request_uri;
    }
    location /c.gif {
        proxy_pass https://c.clarity.ms$request_uri;
        proxy_http_version 1.1;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection "upgrade";
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Real-PORT $remote_port;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
    }
    location /collect {
        proxy_pass https://z.clarity.ms$request_uri;
        proxy_http_version 1.1;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection "upgrade";
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Real-PORT $remote_port;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
    }
    root /var/www/html;
}

CodeBlock Loading...

Reconfiguring the Website

Drawbacks

The drawback after doing this is that the dashboard will show all visits as coming from the nginx server's location. It would be great if Microsoft could determine geographic location from request headers.
It cannot automatically follow Microsoft's clarity.js upgrades, but you can write a script to automate it. Reference code:

PYTHON

import os
import re
import requests
from dotenv import load_dotenv

load_dotenv()

PROJECT_ID = os.getenv("PROJECT_ID")
BASE_DIR = os.getenv("BASE_DIR", "/var/www/html/clarity")
CUSTOM_DOMAIN = os.getenv("CUSTOM_DOMAIN")
NGINX_CONF = os.getenv("NGINX_CONF", "/etc/nginx/conf.d/clarity.conf")
if not PROJECT_ID:
    raise ValueError("PROJECT_ID is not set in the environment variables.")
if not CUSTOM_DOMAIN:
    raise ValueError("CUSTOM_DOMAIN is not set in the environment variables.")

def get_index(try_times = 5):
    resp = requests.get(f'https://www.clarity.ms/tag/{PROJECT_ID}')
    if resp.status_code != 200:
        if try_times > 0:
            return get_index(try_times - 1)
        raise RuntimeError(f"Failed to fetch data from Clarity. Status code: {resp.status_code}\n{resp.text}")
    return resp

def get_script(url, try_times = 5):
    resp = requests.get(url)
    if resp.status_code != 200:
        if try_times > 0:
            return get_script(url, try_times - 1)
        raise RuntimeError(f"Failed to fetch Clarity script. Status code: {resp.status_code}\n{resp.text}")
    return resp

resp = get_index()
script_content = resp.text

clarity_js_url = re.search(r'https://www.clarity.ms/s/[^"]+\.js', script_content)
if not clarity_js_url:
    raise RuntimeError("Clarity script URL not found in the response.")
clarity_js_url = clarity_js_url.group(0)
upload_url = re.search(r'"upload":"([^"]+)"', script_content)
if not upload_url:
    raise RuntimeError("Upload URL not found in the response.")
upload_url = upload_url.group(1)

clarity_js_version = clarity_js_url.split('/')[-2]
clarity_js_path = os.path.join(BASE_DIR, 's', clarity_js_version, 'clarity.js')
if not os.path.exists(clarity_js_path):
    resp = get_script(clarity_js_url)
    os.makedirs(os.path.dirname(clarity_js_path), exist_ok=True)
    with open(clarity_js_path, 'wb') as f:
        f.write(resp.content)

with open(NGINX_CONF, 'w') as f:
    f.write("""server {
    listen 80;
    listen [::]:80;
    listen 443 ssl;
    listen [::]:443 ssl;
    server_name %s;
    if ($scheme = http) {
        return 301 https://$host$request_uri;
    }
    location /c.gif {
        proxy_pass https://c.clarity.ms$request_uri;
        proxy_http_version 1.1;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection "upgrade";
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Real-PORT $remote_port;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
    }
    location /collect {
        proxy_pass https://%s$request_uri;
        proxy_http_version 1.1;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection "upgrade";
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Real-PORT $remote_port;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
    }
    root %s;
}
""" % (CUSTOM_DOMAIN, upload_url.split('/')[2], BASE_DIR))

os.system('nginx -s reload')

new_script_content = script_content.replace(
    'www.clarity.ms',
    CUSTOM_DOMAIN
).replace(
    upload_url.split('/')[2],
    CUSTOM_DOMAIN
).replace(
    'c.clarity.ms',
    CUSTOM_DOMAIN
)

with open(os.path.join(BASE_DIR, 'tag', PROJECT_ID), 'w') as f:
    f.write(new_script_content)

CodeBlock Loading...

PYTHON

import os
import re
import requests
from dotenv import load_dotenv

load_dotenv()

PROJECT_ID = os.getenv("PROJECT_ID")
BASE_DIR = os.getenv("BASE_DIR", "/var/www/html/clarity")
CUSTOM_DOMAIN = os.getenv("CUSTOM_DOMAIN")
NGINX_CONF = os.getenv("NGINX_CONF", "/etc/nginx/conf.d/clarity.conf")
if not PROJECT_ID:
    raise ValueError("PROJECT_ID is not set in the environment variables.")
if not CUSTOM_DOMAIN:
    raise ValueError("CUSTOM_DOMAIN is not set in the environment variables.")

def get_index(try_times = 5):
    resp = requests.get(f'https://www.clarity.ms/tag/{PROJECT_ID}')
    if resp.status_code != 200:
        if try_times > 0:
            return get_index(try_times - 1)
        raise RuntimeError(f"Failed to fetch data from Clarity. Status code: {resp.status_code}\n{resp.text}")
    return resp

def get_script(url, try_times = 5):
    resp = requests.get(url)
    if resp.status_code != 200:
        if try_times > 0:
            return get_script(url, try_times - 1)
        raise RuntimeError(f"Failed to fetch Clarity script. Status code: {resp.status_code}\n{resp.text}")
    return resp

resp = get_index()
script_content = resp.text

clarity_js_url = re.search(r'https://www.clarity.ms/s/[^"]+\.js', script_content)
if not clarity_js_url:
    raise RuntimeError("Clarity script URL not found in the response.")
clarity_js_url = clarity_js_url.group(0)
upload_url = re.search(r'"upload":"([^"]+)"', script_content)
if not upload_url:
    raise RuntimeError("Upload URL not found in the response.")
upload_url = upload_url.group(1)

clarity_js_version = clarity_js_url.split('/')[-2]
clarity_js_path = os.path.join(BASE_DIR, 's', clarity_js_version, 'clarity.js')
if not os.path.exists(clarity_js_path):
    resp = get_script(clarity_js_url)
    os.makedirs(os.path.dirname(clarity_js_path), exist_ok=True)
    with open(clarity_js_path, 'wb') as f:
        f.write(resp.content)

with open(NGINX_CONF, 'w') as f:
    f.write("""server {
    listen 80;
    listen [::]:80;
    listen 443 ssl;
    listen [::]:443 ssl;
    server_name %s;
    if ($scheme = http) {
        return 301 https://$host$request_uri;
    }
    location /c.gif {
        proxy_pass https://c.clarity.ms$request_uri;
        proxy_http_version 1.1;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection "upgrade";
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Real-PORT $remote_port;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
    }
    location /collect {
        proxy_pass https://%s$request_uri;
        proxy_http_version 1.1;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection "upgrade";
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Real-PORT $remote_port;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
    }
    root %s;
}
""" % (CUSTOM_DOMAIN, upload_url.split('/')[2], BASE_DIR))

os.system('nginx -s reload')

new_script_content = script_content.replace(
    'www.clarity.ms',
    CUSTOM_DOMAIN
).replace(
    upload_url.split('/')[2],
    CUSTOM_DOMAIN
).replace(
    'c.clarity.ms',
    CUSTOM_DOMAIN
)

with open(os.path.join(BASE_DIR, 'tag', PROJECT_ID), 'w') as f:
    f.write(new_script_content)