GNU bug report logs - #47276
Stuck at Cloudflare 'browser checks'

Previous Next

To reply to this bug, email your comments to 47276 AT debbugs.gnu.org.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: João Pedro Simas <jpsimas <at> gmail.com>
To: bug-gnuzilla <at> gnu.org
Subject: Stuck at Cloudflare 'browser checks'
Date: Sat, 20 Mar 2021 06:44:52 +0000

[Message part 1 (text/plain, inline)]

In various websites that use cloudflare's 'browser checks', IceCat 78.8.0
running on GNU Guix get's stuck in a loop, in which the page gets reloading
indefinetely.
The particular website I found this issue is gitlab.com, in particular it's
sign in page ( https://gitlab.com/users/sign_in).
I've also found an open issue about it on GNU Guix's bug mailing list (
https://issues.guix.gnu.org/45179) that mentions that at some point this
issue potentialy could be fixed by changing the user agent, but it is not
the case anymore.

Sincerely
João

[Message part 2 (text/html, inline)]

Message #8 received at submit <at> debbugs.gnu.org (full text, mbox):

From: bill-auger <bill-auger <at> peers.community>
To: bug-gnuzilla <at> gnu.org
Subject: Re: bug#47276: Stuck at Cloudflare 'browser checks'
Date: Sat, 20 Mar 2021 14:09:00 -0400

i dont have a solution for this; but it turned up on the
parabola bug tracker some months ago, conflated with other
similar issues, which affected icecat and parabola's iceweasel,
probably since v81 - i dont believe that the problem existed
when iceweasel was v78, so it is probably not the browser
configuration that changed; but something gitlab.com changed
recently

there seems to be multiple reasons why some websites dont work -
one of them is the "enhanced tracking protection", another is
geo-location - relaxing those settings, did not make gitlab.com
work though

there are multiple tickets on the gitlab bug tracker, which seem
to be the same problem - several suggest that it is related to
cookie expiration; but nothing indicated a solution

i discovered that i could make gitlab,com work with iceweasel,
by deleting the profile under ~/.mozilla/; but that trick did
not work for icecat - i still dont know why that website rejects
icecat - other instances of the gitlab software work as expected
- changing the user-agent does not help either

Message #11 received at 47276 <at> debbugs.gnu.org (full text, mbox):

From: Mart Rootamm <martrootamm <at> gmail.com>
To: bill-auger <bill-auger <at> peers.community>
Cc: 47276 <at> debbugs.gnu.org
Subject: Re: bug#47276: Stuck at Cloudflare 'browser checks'
Date: Wed, 24 Mar 2021 23:43:50 +0200

Right-click the GitLab page, click on View Page Info, go to
Permissions tab, scroll to the Cookie section, uncheck the default,
and make sure Allow (radio button) is checked.

In some cases, there's a Tools > Page Info menu option to access the
same page properties window.

Allowing cookies like that has usually worked for most things that
have Cloudflare or other like protection from automated.

Mozilla removed cookie prompting from Firefox 44.0. This worked by
prompting to set cookies before any cookie was saved, and a user could
allow a domain, allow it for session, or block it, and then set the
permission as permanent. This also applied for third-party domains.

A developer named Savarese has continually created patches to
reinclude this functionality:
https://www.savarese.org/patches/firefox.html

If this very important privacy-enhancing functionality were
reintroduced in GNU IceCat (including in older verisons, such as 68.x
for Android), I'd make this browser my daily driver everywhere.

-Mart.

2021-03-20 20:09 GMT +02:00, bill-auger <bill-auger <at> peers.community>:
> i dont have a solution for this; but it turned up on the
> parabola bug tracker some months ago, conflated with other
> similar issues, which affected icecat and parabola's iceweasel,
> probably since v81 - i dont believe that the problem existed
> when iceweasel was v78, so it is probably not the browser
> configuration that changed; but something gitlab.com changed
> recently
>
> there seems to be multiple reasons why some websites dont work -
> one of them is the "enhanced tracking protection", another is
> geo-location - relaxing those settings, did not make gitlab.com
> work though
>
> there are multiple tickets on the gitlab bug tracker, which seem
> to be the same problem - several suggest that it is related to
> cookie expiration; but nothing indicated a solution
>
> i discovered that i could make gitlab,com work with iceweasel,
> by deleting the profile under ~/.mozilla/; but that trick did
> not work for icecat - i still dont know why that website rejects
> icecat - other instances of the gitlab software work as expected
> - changing the user-agent does not help either
>
>
>
>

Message #14 received at 47276 <at> debbugs.gnu.org (full text, mbox):

From: bill-auger <bill-auger <at> peers.community>
To: Mart Rootamm <martrootamm <at> gmail.com>
Cc: 47276 <at> debbugs.gnu.org
Subject: Re: bug#47276: Stuck at Cloudflare 'browser checks'
Date: Wed, 24 Mar 2021 19:29:44 -0400

On Wed, 24 Mar 2021 23:43:50 +0200 Mart wrote:
> Right-click the GitLab page, click on View Page Info, go to
> Permissions tab, scroll to the Cookie section, uncheck the default,
> and make sure Allow (radio button) is checked.

thanks for the suggestion; but it does not work - if it did, i
would expect that it could be made to be the "use default"
behavior for all websites, via the global preferences

that could be by un-checking "cookies" in the "custom" section of
"Enhanced Tracking Protection", or by whitelisting the host in
"Manage Permissions" under "Cookies and Site Data"

even with all three of those, gitlab.com sign-in page is still
broken

Message #17 received at 47276 <at> debbugs.gnu.org (full text, mbox):

From: bill-auger <bill-auger <at> peers.community>
Cc: 47276 <at> debbugs.gnu.org
Subject: Re: bug#47276: Stuck at Cloudflare 'browser checks'
Date: Mon, 10 May 2021 20:07:07 -0400

i may have found a solution for this

there has been a long-standing debate in parabola, as to whether
spoofing the user-agent is a good anti-fingerprinting measure -
people tend to agree that it probably does more harm than good

for example, 'IceCat' in the user-agent string, actually presents
significantly more identifiable information than the generic
'Firefox' would

in addition to the user-agent string, there are javascript
properties, which identify the browser and host - these are
supposedly deprecated now; but they are available - icecat
reports: 'Windows NT 6.1', where i believe this would be
'Windows NT 10.0' on most current windows systems

related to this ticket, i have removed the user-agent
over-rides in parabola's iceweasel, for the reasons above -
the user-agent is now the same as archlinux and most other
distros (the defaults) - in doing so, the problem with login
to gitlab.com was resolved - i discovered that it is the
'general.platform.override' property, which is responsible for
the gitlab/cloudflare rejecting the browser - presumably,
because it conflicts with the user-agent information - setting
it to 'Linux x86_64' (manually) satisfied the browser check;
so this appears to be the general solution

at least, the 'oscpu' and 'platform' properties should be updated
to match the most common windows hosts - 'appVersion' also appears
to be wrong - the default value is '5.0 (X11)' (not the browser
version) - however, changing those properties to agree with the
user-agent string, is effectively removing the overrides (which
again, are deprecated)

here is a test page:
  <html><body><script type="text/javascript">
  document.write("codeName="   + navigator.appCodeName + "<br />");
  document.write("appName="    + navigator.appName     + "<br />");
  document.write("appVersion=" + navigator.appVersion  + "<br />");
  document.write("oscpu="      + navigator.oscpu       + "<br />");
  document.write("platform="   + navigator.platform    + "<br />");
  document.write("product="    + navigator.product     + "<br />");
  document.write("buildID="    + navigator.buildID     + "<br />");
  document.write("userAgent="  + navigator.userAgent   + "<br />");
  </script><body><html>

icecat:
  codeName=Mozilla
  appName=Netscape
  appVersion=78.0
  oscpu=Windows NT 6.1
  platform=Win32
  product=Gecko
  buildID=Gecko/20100101
  userAgent=Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101 IceCat/78.0

arch (default values):
  appCodeName=Mozilla
  appName=Netscape
  appVersion=5.0 (X11)
  oscpu=Linux x86_64
  platform=Linux x86_64
  product=Gecko
  buildID=Gecko/20100101
  userAgent=Mozilla/5.0 (X11; Linux x86_64; rv:88.0) Gecko/20100101 Firefox/88.0

Message #20 received at 47276 <at> debbugs.gnu.org (full text, mbox):

From: Mart Rootamm <martrootamm <at> gmail.com>
To: 47276 <at> debbugs.gnu.org
Subject: Re: bug#47276: Stuck at Cloudflare 'browser checks'
Date: Wed, 12 May 2021 02:26:30 +0300

It appears, that Cloudflare might have been designed to detect a
mismatch between what the string reports itself as, and what the oscpu
and platform strings say.

> whitelisting the host in "Manage Permissions" under
> "Cookies and Site Data"

This whitelisting works via View Page Info, too, where Allow under
Cookies is selected.

-M.

2021-05-11 3:07 GMT +03:00, bill-auger <bill-auger <at> peers.community>:
> i may have found a solution for this
>
> there has been a long-standing debate in parabola, as to whether
> spoofing the user-agent is a good anti-fingerprinting measure -
> people tend to agree that it probably does more harm than good
>
> for example, 'IceCat' in the user-agent string, actually presents
> significantly more identifiable information than the generic
> 'Firefox' would
>
> in addition to the user-agent string, there are javascript
> properties, which identify the browser and host - these are
> supposedly deprecated now; but they are available - icecat
> reports: 'Windows NT 6.1', where i believe this would be
> 'Windows NT 10.0' on most current windows systems
>
> related to this ticket, i have removed the user-agent
> over-rides in parabola's iceweasel, for the reasons above -
> the user-agent is now the same as archlinux and most other
> distros (the defaults) - in doing so, the problem with login
> to gitlab.com was resolved - i discovered that it is the
> 'general.platform.override' property, which is responsible for
> the gitlab/cloudflare rejecting the browser - presumably,
> because it conflicts with the user-agent information - setting
> it to 'Linux x86_64' (manually) satisfied the browser check;
> so this appears to be the general solution
>
> at least, the 'oscpu' and 'platform' properties should be updated
> to match the most common windows hosts - 'appVersion' also appears
> to be wrong - the default value is '5.0 (X11)' (not the browser
> version) - however, changing those properties to agree with the
> user-agent string, is effectively removing the overrides (which
> again, are deprecated)
>
> here is a test page:
>   <html><body><script type="text/javascript">
>   document.write("codeName="   + navigator.appCodeName + "<br />");
>   document.write("appName="    + navigator.appName     + "<br />");
>   document.write("appVersion=" + navigator.appVersion  + "<br />");
>   document.write("oscpu="      + navigator.oscpu       + "<br />");
>   document.write("platform="   + navigator.platform    + "<br />");
>   document.write("product="    + navigator.product     + "<br />");
>   document.write("buildID="    + navigator.buildID     + "<br />");
>   document.write("userAgent="  + navigator.userAgent   + "<br />");
>   </script><body><html>
>
> icecat:
>   codeName=Mozilla
>   appName=Netscape
>   appVersion=78.0
>   oscpu=Windows NT 6.1
>   platform=Win32
>   product=Gecko
>   buildID=Gecko/20100101
>   userAgent=Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101
> IceCat/78.0
>
> arch (default values):
>   appCodeName=Mozilla
>   appName=Netscape
>   appVersion=5.0 (X11)
>   oscpu=Linux x86_64
>   platform=Linux x86_64
>   product=Gecko
>   buildID=Gecko/20100101
>   userAgent=Mozilla/5.0 (X11; Linux x86_64; rv:88.0) Gecko/20100101
> Firefox/88.0
>
>
>
>

Message #23 received at 47276 <at> debbugs.gnu.org (full text, mbox):

From: bill-auger <bill-auger <at> peers.community>
Cc: 47276 <at> debbugs.gnu.org
Subject: Re: bug#47276: Stuck at Cloudflare 'browser checks'
Date: Wed, 12 May 2021 00:38:48 -0400

this is the browser/host info from a real windows10 system running upstream firefox:
  codeName=Mozilla
  appName=Netscape
  appVersion=5.0 (Windows)
  oscpu=Windows NT 10.0; Win64; x64
  platform=Win32
  product=Gecko
  buildID=20181001000000
  userAgent=Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:88.0) Gecko/20100101 Firefox/88.0


here are two mutually-exclusive patches, one for each potential fix:

update spoofed user-agent to match firefox on windows10:
https://git.parabola.nu/~bill-auger/icecat.git/commit/?h=update-spoofed-useragent&id=9b0b4bbbbf144d6d1f319330191ed8990079b6a7

do not spoof user-agent at all (default generic "Linux <ARCH>"):
https://git.parabola.nu/~bill-auger/icecat.git/commit/?h=do-not-spoof-useragent&id=1351954b2fba37cec9262e970a84ce0d7b62df1d

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.