I am trying to understand what NS records are, how glue records form part of it and what happens afterwards?
Glue records are not part of NS records.
Glue records, or glues, are A and AAAA records, and the only thing different is that they are published at the parent side of the delegation instead (in fact in addition of) being published at the authoritative side, which is the children of the delegation.
Most of your questions are difficult to answer because you are not using specific examples. Things depend a lot on using either an authoritative or recursive nameserver for queries, so it is not clear what you are doing.
1.) How does the recursor know if the NS record points to the authoritative nameserver or directs the recursor to another nameserver?
NS records always point to authoritative nameservers, that is their definition.
RFC 1034 §3.6:
NS
the authoritative name server for the domain
(Of course a client has no way to check if that is true or not, in the sense that, without DNSSEC, this could be changed in the path or by the recursive resolver you are querying).
I mean the entries in the NS record are just 'best effort' nameservers
No. I don't know from where you get the "best effort" thing, but NS records are authoritative nameservers and nothing else.
2.) Once the recursor finds an entry in the NS record in a particular nameserver (e.g. TLD nameserver), does it check the A record to find the corresponding ip address or does it repeat the whole DNS process again (querying the root nameserver, then TLD nameserver, etc.)?
A NS record gives a name. That name needs to be resolved to one or more IPv4 or IPv6 addresses, so any user (client) of that, needs to resolve that name to an IP, which happens through the exact same mechanism as anything else, except of course that the cache is heavily used there (and hence major nameservers names are in cache worldwide as used by many domains, so their IP is not often resolved). So, yes, it starts back at the root... except of course in the presence of glues, and when they can be accepted.
3.) How does "glueing" exactly work, I'm aware it's related to avoid recursive queries but are the glueing ip addresses found in the A record? In case of no glueing, does it mean that for 2.) the whole DNS process starts all over again for that hostname (coming from the NS record I guess)?
Simple. Imagine you have example.com domain, which is delegated from com. So .com nameservers have NS records pointing example.com to its nameservers. Now imagine that you choose the "in-bailiwick" situation where the domain nameservers are themselves "under" the domain, aka the nameservers are ns1.example.com and ns2.example.com. Those would be the NS records at parent. But with all that, resolution is impossible. You ask parent "who is responsible (authoritative) for example.com" and you get back "contact ns1.example.com and ns2.example.com" but to do that you need to resolve those 2 names and hence, starting from root again you come again to "who is responsible (authoritative) for example.com" so endless loops.
In case like these there needs to be glues, aka A/AAAA records at the parent, that the client gets back at the same time it gets the NS reply back.
Let us take a real example, with traces as they are now, the domain icann.org.
First, what are .org nameservers?
$ dig org. NS +short
c0.org.afilias-nst.info.
d0.org.afilias-nst.org.
a2.org.afilias-nst.info.
b0.org.afilias-nst.org.
b2.org.afilias-nst.org.
a0.org.afilias-nst.info.
Let us pick any of these and ask for NS records of icann.org:
$ dig @b2.org.afilias-nst.org. icann.org NS
; <<>> DiG 9.18.4 <<>> @b2.org.afilias-nst.org. icann.org NS
; (1 server found)
;; global options: +cmd
;; Sending:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 62392
;; flags: rd ad; QUERY: 1, ANSWER: 0, AUTHORITY: 0, ADDITIONAL: 1
;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 4096
; COOKIE: 4c1ae1af0f0d1e3f
;; QUESTION SECTION:
;icann.org. IN NS
;; QUERY SIZE: 50
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 62392
;; flags: qr rd; QUERY: 1, ANSWER: 0, AUTHORITY: 4, ADDITIONAL: 3
;; WARNING: recursion requested but not available
;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 1232
;; QUESTION SECTION:
;icann.org. IN NS
;; AUTHORITY SECTION:
icann.org. 1h IN NS a.icann-servers.net.
icann.org. 1h IN NS b.icann-servers.net.
icann.org. 1h IN NS c.icann-servers.net.
icann.org. 1h IN NS ns.icann.org.
;; ADDITIONAL SECTION:
ns.icann.org. 1h IN A 199.4.138.53
ns.icann.org. 1h IN AAAA 2001:500:89::53
As expected, the AUTHORITY section gives you the list of NS records. But here you have an ADDITIONAL section. Why? Because one of the 4 nameservers for icann.org is ns.icann.org so it is in-bailiwick. To be able to resolve it, the parent (.org nameservers) have to publish glues too, which are A/AAAA records... and they are in the ADDITIONAL section, hence the client can now contact ns.icann.org immediately without further DNS query, because it has already the IP addresses.
Looking at that, one may wonder why doesn't everyone just use glues and nothing else, as it seems "better" (faster, no dependencies, etc.)?
In fact, it is not better, it has advantages and drawbacks (which is why in most cases, a good domain would use some in-bailiwick nameserver and some out-of-bailiwick).
Among drawbacks:
- the IP addresses have to be managed at the registry; if they change, a change needs to be done (outside the DNS, typically with EPP) by the registrar of the domain on behalf of the client; this is often forgotten and creating lots of problems
- even if
icann.org is fully DNSSEC enabled, the reply from .org authoritative nameservers does not sign the records in the ADDITIONAL section, which means they can be changed on the path without the change being detected
4.) In Cloudfare's explanation for the NS record, the example includes an @ and I've seen it too in this example Zone DNS, Example Zone File , what does the @ or leaving that column blank actually mean?
Both are not really part of the DNS, but conventions called "master zonefile format", loosely defined in §5 of RFC1035:
The last two forms represent RRs. If an entry for an RR begins with a
blank, then the RR is assumed to be owned by the last stated owner.
and
@ A free standing @ is used to denote the current origin.
It just means, that if you are writing the zonefile for zone example.com for any record you can either write example.com. (here the final dot is very important, if you forget it, it means relative so a name of example.com in this file really means example.com.example.com.) or @ it is the same thing, so up to you to decide which you choose. Why is it useful? Because for some generic zones, you can have the exact same zone content for multiple zones. If you write the zone content with @ and hence do not have to write down the zone name inside the zonefile you can reuse the same file for multiple zones, simplifying maintenance.
As for the "leaving the column blank", again it is a shortcut you can use or not.
The following two are the same:
foobar IN A 192.0.2.42
foobar IN TXT "I am foobar"
and
foobar IN A 192.0.2.42
IN TXT "I am foobar"
(amount of spaces is just for alignment, no specific amount is needed outside of at least one)
or even:
foobar IN TXT "I am foobar"
IN A 192.0.2.42
You have less to write in the second case, but with the drawback that you can NOT any more move the lines around easily, as the second line needs the context of the first one.
This is an implementation detail of the first nameservers (bind), and depicts how the content of the zone is written in so called textual zonefiles. Other nameservers have to follow that convention and support the same format.
But you never do a DNS query for @ or for an empty name (that case is not even possible). So all of that is hidden inside nameservers.
