Absolutely possible. Really do need to know how many people you want to support. A typical Medieval town can range greatly in population, but this could certainly support a few hundred people. Take a look at the answers to this question right here on stack exchange.
As long as you have the basics, food, water, and all that it's certainly possible.
Because it's right by a mountain range, water might actually flow through this area, and it's even better if you have a spring.
Just because an area is desert doesn't mean that there isn't a water supply present. They should be able to grow or trade for food.
The mountains are really the only difference between your question and the other (which is actually marked Medieval in the tags).
To answer the population question, take a look at this question, which is about how far apart Medieval cities tend to be, but my answer there covers population as well. Keep in mind that the conditions you outline would mean that it might be far from other cities/townships but the next small town could be situated in the valley through the pass. Because conditions for farming aren't as good, naturally your town is going to be further away from other places--more like the 10 mile limit in Giant Cow's answer. And there probably won't be anything out further if there's a lack of resources.
The other thing you might want is...something on the other side of the desert that people want to get to. It's a good reason for your outpost to exist. That something can be trade with another nation or something else entirely.
Desert conditions vary widely--so it doesn't have to be your classic Sahara-- or it can have a transition to different types of desert. See this link for more info.
1Yes of course. For example if it holds religious significance- even if it has no economic benefit and can't produce a lot of food because it's right next to a desert. The real question is how big it can get. More information is needed about the location and so on. Btw if you've got mountains and a desert, where is the empire? – Raditz_35 – 2018-01-30T19:09:05.933
1The questions you should ask yourself, because your readers will, are: why would it be built there? Where does water and food come from? – Mołot – 2018-01-30T19:26:09.390
1Samarkand? It's ancient, it still exists, and it's situated at the foot of the Pamirs... – AlexP – 2018-01-30T19:30:19.810
2this may sound like a strange question but how does a town "face" something? – John – 2018-01-30T19:39:27.327
Kashgar, Taraz, Peshawar, Herat, Isfahan, Dire Dawa, Tinghir, Lanzhou, Salt Lake City, El Paso, Mendoza... – kingledion – 2018-01-30T19:45:44.420
1As well as trade and an outpost as people mentioned, there are all the normal reasons a town may be built (e.g. a mine nearby? Strategic reasons?) as for how to get water to it: aquaducts (coming from the other, wetter side of the mountain)! – colmde – 2018-01-31T13:02:54.527
@John In this case, "facing" is referring to the side of the mountain, not a direction the town is pointed, it can probably just be removed – phflack – 2018-01-31T17:40:06.197