Blood from a silicon turnip


Now that we all have the hangover from hell after the big OpenPOWER-is-open party and are sitting around nursing headaches and sipping raw eggs from brandy snifters, let's talk about squeezing blood out of silicon turnips.

In general my cursory view of the Internet demonstrates two, maybe three, reactions to the OpenPOWER announcement:

"Hey, cool!" (or, less commonly but frequently enough to be obnoxious, "Didn't PowerPC die years ago?")

and

"It's too expensive."

Uniformly these two statements are being said by individual developers talking about getting one of their own systems, at least publicly, anyway (enterprise customers may also be complaining but I haven't seen very much in the places that are publicly visible). For the sake of the discussion let's ignore both the fact that people who skimp on privacy and owner control for a cheaper system are slowly boiling themselves alive in their own cauldrons, and the fact that you can go get a (often substantially) cheaper Intel or AMD system and have similar performance if not better because the CPU optimizations already exist.

The problem really isn't the CPUs. You could cheap out and buy some lower binned consumer part, and I'm sure some of you are very happy with those, but realistically POWER9 is meant to complete against server-grade tiers. At the Cascade Lake level, Intel's most similar 16-thread part is the 8-core Xeon Silver 4209T, with 11MB of L3 and clocked from 2.2 to 3.2GHz for $500 MSRP, or you can go Coffee Lake and get its 8-core/16-thread part, clocked between 3.7 and 5.0GHz and with 16MB of L3 as the E-2288G for about $540 MSRP as of this writing, though the E-2288G also has a GPU. AMD has a 16-thread Rome Epyc (the 7232P) with clocks from 3.1 to 3.2GHz and 32MB of L3 for about $450 MSRP. I think we can agreeably stipulate that both of those are ballpark comparable with a Sforza 4-core POWER9, also 16 threads, with 40MB L3 (10MB per core, unpaired); Raptor is the only retail source for this right now and they sell a 3.2-3.8GHz clocked part (CP9M01) for about $440.

As for a 32-thread Xeon, Intel doesn't sell a 32-thread Coffee Lake. You'll have to buy Cascade Lake, and your closest option is the 16-core/32-thread Xeon Silver 4216, also 2.1-3.2GHz, with 22MB of L3 for $1000. AMD offers the 16-core/32-thread 7302P, 3-3.3GHz and 128MB of L3, for $825 MSRP. Raptor, again, is the only retail source for the 8-core/32-thread POWER9 and they sell a 3.45-3.8GHz clocked part with 80MB L3 (CP9M02) for $690. In fact, let's be ridiculous and comparison-price the 22-core, 88-thread monster. Raptor sells this 2.75-3.8GHz part with 220MB L3 (CP9M08) for $2800. Coffee Lake, sir? Sorry, sir. Intel does list a Cascade Lake Xeon Platinum 9242 with 48 cores, 96 threads, 71.5MB of L3 and clocks from 2.3 to 3.8GHz, but the MSRP for such systems is atrociously high (estimated north of $25,000). The closest Epyc is probably the 48-core/96-thread 2.2-3.3GHz 7552 with 192MB L3; even that will set you back $4025.

Not only can we conclude that POWER9 CPUs are reasonably priced, but I think there's also a credible argument that they're competitively priced. There's a reason for this: Raptor doesn't make them. They're shipped in from IBM's supply chain (presumably from GlobalFoundries) and IBM not only has them made in volume, but higher-cored parts where not all the cores are working can be binned lower for this market and increase overall yield, thus improving the economy of scale.

All right, so what about the logic boards? A quick survey of LGA 1151 (Coffee Lake) server-grade boards on Newegg averaged around $250 and LGA 3647 (Cascade Lake Gold/Silver) around $400, with varying numbers of expansion and RAM slots, though I have no idea what a BGA 5903 board for that Xeon Platinum part would run. SP3-socket boards for the Epyc look comparable. Meanwhile, the cheapest Raptor motherboard (as an item) is the basic Blackbird starting at $1100. Is this justified?

As it happens, we actually do have other PowerPC small-volume systems to compare against. They're called Amigas, or at least the AmigaOne. Even as an Amigaphile I have never been shy about voicing my displeasure with their running embedded parts as entire systems and the P5020 they're using in the current X5000 is basically at a G5 level of performance (until you factor in its loss of AltiVec, and then the G5 stomps it on such tasks), but they're out there and you can buy one. At £1800 from AmigaKit, that's about US$2200 right now prior to Brexit, plus shipping. It includes the case, Radeon GPU, 2GB RAM, optical and spinning disk, CPU and board. Ignoring the obvious performance differences, I paid about $2100 for my 4-core Blackbird system with everything there minus the GPU, but more RAM and an SSD. (By the way, the parents were visiting not too long ago and we watched Glove and Boots videos from YouTube on the home theatre with it. Worked fine. I may not install a GPU in it after all.)

We can extrapolate prosumer pricing too. While I couldn't find my Quad G5's original sales receipt, I seem to recall that I paid around $3600 in 2006 for it (4 cores, no SMT), 4GB RAM, a hard disk and an ATI 7800GT video card. That's about $4500 in current money for a system that was not massively high volume, but not particularly niche, and could be readily bought at the consumer level. IBM provided the chips for that too. Currently the Talos II with a single-4 (16 threads), 16GB of RAM, 500GB NVMe and a WX7100 is selling for $6500, despite in much smaller volumes than the Quad.

This is to say that Raptor's pricing is by no means out of whack for boutique low-volume sales, and again, arguably even competitive. Let's remember that first and foremost Raptor is a small company. Many of the people coming new to the platform don't remember the original POWER8 Talos crowdsourcing attempt, but I do, because I was one of the people who had my money in. They needed about $3.7 million to do the job and unsurprisingly that went aground as you might recall, but Raptor refunded people's money and this went a great deal to establishing their trustworthiness. As such, I imagine there was no small amount of internal investment required to launch the Talos II (which I was delighted to preorder as soon as I could do so). Even though the T2 (and the T2 Lite and Blackbird, by extension) is strongly based on the Romulus reference platform, that doesn't mean there wasn't any R&D required on their part, and there is still manufacturing, QA and support costs as well as the need to actually turn a profit. I mean, seriously, some of you actually seem to expect Raptor to sell these things at a loss. How long do you think they'd stay in business?

Now, with all that said, none of you who have bought one (or several) of these systems will need any convincing that the price is worth it, and it won't convince those of you who have heard these arguments before and discount them. This is a fair criticism because frankly there's no getting around the sticker price, even if I think I've made the case that Raptor cannot easily make it cheaper. So how will they ever get cheaper?

Raptor has a more or less natural monopoly on the OpenPOWER workstation market. Don't get me wrong: I am not accusing them of gouging. As monopolies go, this is about as benign as you can get because not only are they good stewards of the ecosystem but frankly they were simply the first ones in the pool. Look at the OpenPOWER membership list. Do you see anyone else catering to workstation users? (I nearly choked on my Mr Pibb when they talked about Raptor's "low end" systems at the Summit. This is, of course, purely by comparison.) There are some people running some of the Tyan POWER8 systems as workstations but they are clearly not designed as such, and the effect is not unlike running an Xserve G5 instead of the regular Power Mac tower. My POWER6 may be a "tower" system but I sure wouldn't want it under the desk. No one else makes OpenPOWER workstations. No one is even talking about it.

Raptor management may not like me saying this, but this is an independent blog, and if the OpenPOWER workstation market is going to grow and stabilize then there's going to have to be someone else. It's not a situation like the Mac clones where all Umax and Power Computing did was eat Apple's lunch (which is why Steve Jobs canned the whole thing), because Apple was big enough to saturate the market such that anyone who wanted a Mac had one and thus all the clones did was steal sales. By contrast Raptor is not big enough to saturate the OpenPOWER workstation market because they can't move enough units: there is pent-up demand waiting for the price to come down, and they can get backordered even on the systems that people do purchase. Yes, I'm hopeful that an open ISA will lead to new and more exciting chip designs, but as far as the actual cost of the chips themselves, the "big reveal" probably changes the retail cost of the actual CPUs very little if any because they were never priced out of the market to begin with. Where we need improvement is in the cost of the actual systems so that people can get them and there can be more of them. And Raptor cannot do this by themselves.

I like Raptor because I like their people, I like their products and I like the way they do business. If someone else entered this space I would probably still buy from them. But someone else in that space also means new ways of looking at the market, presumably newer niches to distinguish themselves, and hopefully more investor interest in the sector to increase the available capital needed to enable volume production and sales in a way that would actually then start lowering prices.

Plus, more players in the workstation market also means market resiliency. Raptor seems to be a stable company, but what if they weren't? What would we do if they had to close their doors? CPU manufacturers back in the dark ages (around 1978) had to have second sources to get design wins. We need second sources to make the market survive a loss of the primary manufacturer, however unlikely that would be, because their exit with no replacement would doom the Talos family to being a modern Power Mac G5: a dead end.

In the meantime, we want Raptor to do well and their success to attract other players to this market, because right now they're the only (though best) game in town. The cost for the CPU is reasonable, and criticism of their logic board pricing is unjustified, especially as you reach higher core counts (the same Talos board takes a single-4 or a dual-22). If people believe that a non-x86 system is valuable to have, if people (also) believe that supporting an open architecture is valuable to do, and if people (also) believe that a truly owner-controlled workstation is valuable to use, then people need to understand where the market is now and put their money where their mouths are. Best of all, you're not buying an underpowered conversation piece; you're getting a competitive system you can actually use. I don't see how we get much more blood out of that silicon turnip otherwise.

Comments

  1. Do you think the microwatt is capable of competing in the Arduino and Raspberry Pi SoC space?

    ReplyDelete
    Replies
    1. Not in its current form because it's too basic right now, and not particularly high performance. After all, it was written to be easy to understand and to get an MVP in time for the Summit. But it will improve.

      Delete
  2. Well, then... What are your thoughts on the PowerPC notebook project? I help them out with translations and everyone is working very hard for it to see the light of the day. It will not be a wonder monster of performance among laptops but it is being built around the same philosophy as the Talos.

    Shiunbird

    ReplyDelete
    Replies
    1. Although I think there's promise, I'll reserve an opinion until an actual prototype exists. I don't think it would occupy exactly the same market segment though.

      Delete
  3. As with the success of anything in the desktop space, people need to be able to game on it if it's to succeed. People will gladly drop $2500.00 if it will produce 4K high frame rate graphics that will blow their minds. The reason both Microsoft and Intel have such a perennial stranglehold on the desktop market doesn't have anything to do with productivity software, its all about the DirectX API and the sweet, sweet FPS a highly clocked single thread on an Intel chip could provide. AMD has only started to look viable since Zen has started closing that 10% performance gap with Intel and modern game engines have gone multi-core/multi-thread beyond what Intel sells. Linux on any platform has only become interesting since Valve started releasing the Proton shim for older games and game engines have started to bake in Linux support in their newest releases (WINE on its own was never going to bridge that software compatibility gap). Macs during the Jobs era were doomed not just because of lack of an enterprise strategy, but more importantly because Jobs never valued gaming to the point that Gates was able to swoop in and poach the Marathon team to build Halo for Microsoft. Anyway, that's my two cents on the topic.

    ReplyDelete
    Replies
    1. I agree with you regarding the general idea that computer gaming is important, and shouldn't be ignored. I think Unreal Engine 4 works natively on PPC64. Raptor has some videos of that, on the earlier POWER8 Talos.

      Delete
  4. Just a few quick comments. 4k gaming works perfectly if you can get a Power build:

    https://twitter.com/klauskiwi/status/1164273128588972032

    This is just my opinion, but I wonder if a second manufacturer (while important from an ecosystem perspective) would actually lead to higher prices right now. If you assume a steady (growing, but maybe more slowly than I'd like) demand, and suddenly that demand is split between two vendors, both suddenly see half the volume and have to raise prices accordingly. At this stage my concern is that a spike in pricing could be fatal to the nascent Power desktop market.

    Finally, Raptor stated at the Summit that Blackbird is actually a de novo design -- it has no real roots in Romulus, it's all Raptor. That also means it doesn't have Romulus "quirks" like closed firmware SAS controllers, since it was designed to be 100% blob free on the desktop.

    ReplyDelete
    Replies
    1. Well, I said "by extension" for a reason. I think we agree that the T2 (and Lite) are strongly based on it, and the Blackbird is certainly different in many respects including form factor. However, I don't think the Blackbird could exist (technologically, as well as a product) without Romulus because it set the pattern. The block diagram is still very similar.

      I think a split demand situation would only occur if there is no additional unmet need, and I don't think this is the case. Otherwise, it's basically the Mac clones all over again.

      Delete
  5. One thing you have to keep in mind is that POWER9 cores are not that competitive in raw computing power. I see the 4 core low entry POWER9 CPU competing with an AMD RYZEN 2200G or 2400G sans capable 3D graphics of course. I did a comparison today with results from Michael's June article https://www.phoronix.com/scan.php?page=article&item=spec-power9-4core&num=1 with my own RYZEN 2200G so if you're interesting you will find results here: https://openbenchmarking.org/result/1908271-HV-1906038HV44
    This is - as in all comparisons - a bit unfair acknowledging decades of x86 compiler optimizations not found in the POWER world but from a desktop user POV an important point.

    ReplyDelete
    Replies
    1. I don't understand your statement. Which is it, the compiler or the chip? You make an allegation it's underpowered and then you add that the comparison is unfair, so I'm not sure how much of your statement to take at face value.

      Delete
    2. PTS has known problems as well with Power, specifically relating to projects like LAME not even passing in correct compile flags. You also need to be careful of AMD cheating just like Intel does on benchmarks, the Power systems ship protected against Spectre and Meltdown while the x86 systems often are not.

      Fundamentally it's also not fair to compare a 14nm owner controlled device with a 7nm device that AMD literally holds a master key for at all times. If you are OK with key escrow (so to speak, there's some nuance around this), I'm not entirely sure why you would want or really even need local compute. You can get much cheaper pricing putting your data in a Chinese cloud, and Stadia and such let you rent your games on demand.

      Delete
    3. FUD aside, Ryzen 2200G/2400G is built on 14nm process, not 7 nm. And it is a node inferior to the IBM one, because AMD uses basically Samung-licensed 14LPP bulk process, whereas Power9 is made on a separare high-performance SOI node developed by IBM.

      And Ryzens aren't vulnerable to Meltdown like Power9/8 and Intel so they don't need to mitigate that. Spectre is being mitigated on all OSes by default, so the "but but but unmitigated!" argument should IMHO only be raised once you actually find something to the contrary.

      Delete
    4. If you're comparing similar process (14nm to 14nm), you should find Power is stronger on non-SIMD and AMD is stronger on SIMD. Any differences from that general trend are likely to be software optimization issues.

      Delete
  6. I wish Raptor stopped with blatant FUD like this https://twitter.com/RaptorCompSys/status/1161367369274540033
    They are going too far, this is close in bullshitry to the claim that x86 will stop running Linux and be Windows only, which their guy uttered in Phoronix comments.

    Ethics...

    ReplyDelete
    Replies
    1. What a strange definition of ethics. You defend a company that has forced its customers to accept a total loss of control, and one that won't even provide a basic legal guarantee of PSP firmware correctness.

      You say FUD, I see a certain degree of accuracy. Can you explain exactly how I can remove this unwanted AMD PSP from my Ryzen system?

      Delete
    2. Two things:
      1) The Twitter comment doesn't say nor imply x86 will stop running Linux, which is what you made it sound like;
      2) Absolutely 100% of what is said in that tweet are scientific facts. To call it "bullshitry" is factual ignorance.

      Delete
    3. Aparently you missed the fact that the Linux comment was posted on Phoronix forums.
      But you would then probably just say "hey, it is not impossible, can you prove his prediction was wrong" or something like that.
      Look up the definition of FUD. And try to be less pathologically religious about computers.

      Delete
    4. I think this is where I blow the timeout whistle and politely ask folks (no one in particular) to step back from this comment thread.

      Delete

Post a Comment

Comments are subject to moderation. Be nice.