0
I am in the process of parsing machine-readable data from wiktionary into a SQL database using wikokit, but have since realized that, according to the time estimate provided by wikokit, it will take a solid month to finish (53499 minutes.) This is after following general recommendations for increasing performance on MySQL, including using mysqltuner and following its directions.
I am currently using my personal computer to parse Wiktionary, the specs of which are 4.2 Ghz CPU and 8G ram, using a 1TB HDD. Very naively calculated, I imagine a 256G RAM server would then be able to parse Wiktionary in about a day. I think I need to rent a server high in RAM, average in CPU, and minimal in HDD space, at a per second rate for a day or two. However, I'm not sure whether that would even work (would scaling up memory by 30x really help?), or where to find such a thing, or what to even google to start.
Some more details: Wikokit works by taking a raw, unparsed SQL database of Wiktionary, parsing it through its Java code using Connector/J, then loading it into a formatted SQL database that starts off empty. Essentially raw_enwikt -> wikt_parser.java -> parsed_enwikt. The guide can be found here.
I'm not very experienced in server usage, so I am also unsure of whether you would be able to set up MySQL and run Java code on it on any arbitrary server. I need the parsed database in relatively short time, within the next 3-4 days.
What is the model number of your CPU? (We need more information than just the clock speed like threads, etc.) – zandermar18 – 2019-07-26T20:55:17.553
This question does not appear to be about computer software or computer hardware within the scope defined in the help center. Amazon and Azure both can be used for computational tasks. However, 256 GB virtual machine, will be extremely expensive. You might be better off building your own server, – Ramhound – 2019-07-26T20:55:24.130
Should I ask in server-fault? I posted here because I am not a professional in IT by any means and was hoping personal MySQL use is covered under SU. The CPU is i5-4670k OC'd to 4.2 Ghz. – stripedneck – 2019-07-26T21:07:50.523
To put it into perspective a 64 thread 256 GB Azure VM would be $4.5K per month. You could build a similar server for the same price. – Ramhound – 2019-07-26T21:22:33.337
Server Fault will not make service recommendations. There isn’t a SE for that type of question. Your personal computer is indeed within scope, but your question, isn’t about your current configuration it’s about if a VPS would be appropriate for this task (a matter of opinion) – Ramhound – 2019-07-26T21:24:47.027