0

I had Cassandra cluster of 3 nodes. I was performing some colud migration activity and for that, I added two more nodes in an existing cluster and the result is as follows.

Datacenter: datacenter1
=======================
Status=Up/Down
|/ State=Normal/Leaving/Joining/Moving
--  Address      Load       Tokens       Owns (effective)  Host ID                               Rack
UN  192.168.1.5    24.07 GB   256          59.4%             804b855f-78f3-42d6-8abf-b9aec73699df  rack1
UN  192.168.1.6    24.77 GB   256          59.8%             21f2066f-1794-485c-9c4f-d6d1b286a551  rack1
UN  172.16.2.20  15.96 GB   256          60.3%             2c2f512d-5743-4632-a4b5-cd2cac967897  rack1
UN  172.16.2.21  12.76 GB   256          60.0%             657ff1b6-773a-4782-a506-c4899cdf2a4f  rack1
UN  192.168.1.7    17.69 GB   256          60.5%             c8c4bc41-4b5c-41e6-bb71-ab90c2ed5eb0  rack1

The OWNS field was previously 100% for all nodes and now it shows different numbers so is it like whole 100% data is not on each node and suppose if I take down any node by shutting down Cassandra then is there a risk of data loss?

innervoice
  • 21
  • 5

1 Answers1

1

That's a normal situation.

Most likely you had a replication factor 3, and with 3 nodes this lead that every node had a replicas for all other nodes. When you added 2 more nodes, every node will keep 3/5th of your data - its own range (1/5th of data) + replicas (2/5th of data). It's not exact 60% because the data may not be exactly split between nodes - some partitions could be bigger than other, etc.

I recommend you to read DataStax Architecture Guide - it will give you better understanding how Cassandra works & how data is replicated.

Alex Ott
  • 316
  • 1
  • 5
  • Thanks a lot. It really helped. Now I have set replication_factor to 5 and it shows 100% in OWNS column. Will you please help with the following question. 1. If I directly take down Cassandra process on any machine then will it cause data loss? I have to remove two old machines from this cluster. – innervoice Sep 30 '19 at 19:23
  • It doesn’t need, the RF=3 is standard way, as RF=5 is overkill – Alex Ott Sep 30 '19 at 19:24
  • So suppose If I take down two machines directly and then again set RF=3. Will it work? It won't result in data loss? – innervoice Sep 30 '19 at 19:26
  • But for what purpose? Main purpose of Cassandra is to be distributed - it’s not required that every node kept all data. – Alex Ott Sep 30 '19 at 19:27
  • My old cluster is on azure and now moving it to Google cloud. I'm new to Cassandra and by reading some basic documents I mistakenly added two Google machines to existing cassandra cluster on azure. So wanna come out from this situation without any data loss. :-) – innervoice Sep 30 '19 at 19:31
  • Just change replication factor to 3, run repair on all nodes, and then run nodetool decommission on both google nodes – Alex Ott Sep 30 '19 at 19:32
  • Let us [continue this discussion in chat](https://chat.stackexchange.com/rooms/99311/discussion-between-innervoice-and-alex-ott). – innervoice Sep 30 '19 at 19:33
  • Usually the expansion to another cloud is done via addition of new data center. I recommend to take DS201/DS210 courses on academy.datastax.com – Alex Ott Sep 30 '19 at 19:34