From your description it sounds like the primary use case for the server will be to stream videos to 100 clients.
Given this, the two most important sub-systems for the server will be:
- The network connection
- The storage system
To size these for performance, you need to know how much bandwidth you require - this depends on the quality of the video you will be serving. Assuming that the video is 1MB/s, for 100 users both the network and storage systems must be able to deliver 100MB/s (800Mb/s for the network) of data.
If the network bandwidth connection is greater than a single link can handle and you can't afford to increase the link speed (i.e. migrate from 1GbE to 10GbE/40GbE) then you can add more links and either aggregate them at layer 2 or use multiple IP address (one per link) and distribute the connections through the application. Also you must make sure that the data path from the server to the clients can handle the capacity, there's no point in the server being able to deliver 10Gbps if there is a bottleneck elsewhere in the network!
For the storage system, you need to make sure that it can handle both the bandwidth and the number of IOPS (IOs per second) to serve the data. From our calculation above, you will require 100MB/s of streaming reads, but because you have 100 users, the storage system must be able to handle at least 100 IOPS. If you expect the users to navigate the videos you will need much more to maintain the performance. Whilst hard drives can deliver good streaming performance when they don't have contention, I would consider SSDs when the number of concurrent accesses increases.
Beyond these basic performance characterisations, you also need to consider reliability for both the network connections and storage systems. What happens in the case of link failures or hard drive failures? If you want to continue providing service, you must have multiple links and resilient storage configuration (through RAID or data replication at application/file system level).
RAM for the storage system will help you cache the videos and take some load off the storage system, so if you expect many people to view the same video then more RAM will generally help. But if the size of the working set (the data commonly accessed) is greater than the amount of RAM on the system, don't be surprised if you don't see a great improvement.
Because the majority of the system will be delivering data via IO, you probably don't need much CPU power and so any modern multi-core CPU should be sufficient. Although you should take care that if you have multiple CPUs in a NUMA architecture, you don't overload the CPU interconnect.
Once you have the hardware, you should benchmark it to make sure it delivers performance wise. For the storage system I recommend the fio tool, it allows you to specifically test scenarios such as streaming reads with 100 concurrent readers.
Given all I've said above, you should talk with your preferred vendor and see what configuration they recommend. If you make your requirements part of the RFP (especially if you include hard requirements such as the ability to deliver a particular IO profile) then it can make it easier to resolve any failings in performance later.
When designing these systems, try and think about how the data moves through the system, and where the limitations will be.
Finally, make sure you design the system with excess capacity so you can handle both growth and peaks in traffic.
