Why Our Data Differs from Other Transparency Projects

Why our data sometimes differs from other censorship transparency projects is usually a question of measurement design, not necessarily a contradiction. A blocked website can produce a DNS error for one connection, an IP address that refuses traffic for another, and an HTTP 403 for a third. Each outcome may indicate filtering, but they do not describe the same layer of the network or the same population.

Net Neutrality Monitor focuses on censorship and filtering by Internet Service Providers and DNS servers. It combines submitted URLs and IP addresses with blacklist, probe, DNS and statistical data, then presents country-specific findings. Because the project is a beta, volunteer-supported service developed and powered by AirVPN, its outputs are best read as indicative research observations. Comparing them with another dataset is most useful when the observation conditions are aligned.

Different Measurements, Different Questions

One project may ask whether a URL can be fetched from many networks. Another may ask whether a particular ISP’s DNS server returns an address, or whether a host blocks an IP. The unit under test changes the question: a domain, hostname, resolved IP, port, URL path and protocol can each behave differently. A submission for a website is therefore not equivalent to a submission for one address behind it.

Censorship may occur before routing: recursive DNS resolvers can return NXDOMAIN, SERVFAIL, a sinkhole address or a false positive. It may then appear during transport, where a TCP connection or TLS handshake is reset. Finally, a server can reject an HTTP request while the network path itself remains intact. Calling all three “blocked” hides meaningful technical distinctions.

Coverage creates further differences and potential sampling bias. A list of popular sites can miss niche or newly blocked services, while targeted submissions can overrepresent a campaign or a news event. Results also depend on IPv4 versus IPv6, cached DNS answers, security software, VPNs, proxies and the city from which traffic leaves Australia. A national percentage may conceal this variation.

What the Monitor Measures

Net Neutrality Monitor publishes blacklists, probe results, DNS information and statistics rather than presenting every event as one universal verdict. This separation helps reveal whether an address appeared on a recorded filtering list, failed a probe or behaved differently through DNS. It also lets researchers compare the underlying signal instead of relying only on a final label such as “blocked”.

Users can submit a URL or IP address for monitoring and then review country-specific censorship results. The distinction matters because the same domain can resolve through an Australian ISP while its address is unreachable from selected networks elsewhere. A result tied to one resolver or route should not be generalised across every Australian connection without corroboration.

For network operators and researchers, repeatable testing can be as valuable as a manual lookup. The article on own servers shows how an API can automate censorship checks from infrastructure they control. Such local checks can reduce uncertainty about their own DNS and routing, although they still measure the vantage point where the check runs.

Comparing Measurement Approaches

Measurement methods diverge across several dimensions. A distributed probe can approximate user experience, while an ISP resolver reveals filtering closer to the subscriber. DNS-over-HTTPS can bypass an ISP’s visible resolver path, whereas ordinary DNS may expose it. Comparing the outputs requires checking the target, vantage point, protocol and timing together.

Dimension	Net Neutrality Monitor	Common approach elsewhere	Why results can differ
Target sample	User-submitted URLs and IP addresses, plus blacklist, probe, DNS and statistical data	Broad URL lists, continuous probes or national summaries	Samples cover different sites, addresses and traffic types
Network vantage	ISP and DNS-server observations with country-specific results	Consumer devices, volunteer nodes, cloud servers or remote probes	Routes, peering and resolver policies vary
DNS context	DNS data is published alongside filtering evidence	DNS may be treated only as a prerequisite for a test	Cached, filtered, DoH and DoT answers can differ
Failure signal	Address, domain, blacklist and probe records	Reachability, HTTP status codes or aggregate flags	DNS failure, sinkholing and origin rejection are not equivalent
Protocol coverage	Domain and IP submissions can expose name- and address-level effects	Tests may prioritise HTTP, TLS, IPv4 or IPv6	A path can work under one protocol and fail under another
Time and retries	Each result reflects its recorded observation context	Snapshot, periodic scan or live-test schedules	Transient outages and retries alter apparent blocking
Interpretation	Beta, volunteer-supported and indicative	Each project applies its own definitions and baselines	Labels may describe different technical events

Read across a row rather than comparing totals alone. A DNS discrepancy may reflect the choice of recursive resolver, while an IP discrepancy may come from regional routing or an overloaded origin. Likewise, a country-level count is not directly comparable unless the tested targets overlap. If only a small set of URLs is shared, confidence intervals and percentages can exaggerate what is, in reality, a few divergent observations.

Some differences are deliberate improvements rather than errors. Recording DNS and probe evidence separately gives more context than a single inaccessible-page flag. Other projects may prioritise breadth, independent vantage points or policy classification. Neither approach is automatically superior; each answers a narrower operational question, and their strongest value comes from transparent comparison.

Checks for Comparing Any Dataset

Local context is essential for Australian readers. Sydney and Melbourne hosts do not necessarily share routes, resolver configuration or peering, and traffic from Perth or Adelaide may take a different international path. Australia’s broadband market also includes large providers such as Telstra, Optus, TPG and iiNet, alongside mobile networks that may use different filtering systems. A result from one connection is therefore evidence about a provider and path, not an automatic verdict on all Australian internet access.

Before combining records, use a compact verification routine. The first set of checks establishes whether the tests themselves were equivalent.

Consistency checks

• Does the hostname, port, URL path and resolved IP match?
• Are plain DNS, DoH and DoT treated as separate tests?
• Does the run use the same ISP, city, VPN state and IP family?
• Were retries and peak or off-peak windows consistent?

Those checks identify technical mismatches. A second set preserves the context needed to interpret a disagreement rather than collapsing it into a binary result.

Context to retain

• Was the interference at DNS, TCP, TLS, HTTP or application level?
• Which resolver, ISP, probe location and observation date applied?
• Was the target tested as a domain, an IP address or both?
• Do independent probes or adjacent measurements corroborate it?

Australian legal and market references deserve equal care. ACMA publishes information about blocking notices and the internet censorship regime, yet a national policy summary cannot establish what a particular Australian household observed. Local browsing customs—using mobile data during a fixed-line outage, security applications or a commercial VPN—can also change the path. Report those conditions where known, and mark them unknown when they were not recorded.

A Reliable Interpretation Workflow

A sound comparison begins with the target and ends with the observation, not with the conclusion drawn from either dataset. Preserve the exact hostname and address, note whether DNS succeeded, and separate network failures from an origin server’s response. This sequence helps distinguish filtering from censorship performed by a website, content moderation, an expired certificate or an ordinary outage.

Next, align the network context: resolver, provider, city, protocol, IP family, proxy status and time window. The project documentation can clarify what the Monitor records, how submissions are handled and the intended limits of its research data. Checking those fields prevents an undocumented assumption from becoming a supposed contradiction between projects.

Finally, look for corroboration across independent evidence. A DNS result from one resolver should be checked against an IP probe; a single timeout should be retried from another vantage point where feasible. Do not merge unlike observations into a score. Keep DNS, blacklist, probe and statistical results distinguishable, and report coverage gaps alongside any apparent discrepancy.

The practical takeaway is to compare the measurement setup before comparing the numbers: a matched target, network vantage, protocol and observation window turns conflicting censorship findings into useful evidence about where and how filtering occurred.