While I was at the Data Scotland conference in Edinburgh on Friday (great event by the way) I stopped by the Tabular Editor stand and got the nice people there to give me a demo of their new tool, DAX Optimizer. It’s currently in private beta but if you’re curious to learn more, Nikola Ilic has already blogged about it in detail here.
Rather than blog about the tool itself – there’s no point repeating Nikola’s post – I thought it would be good to answer a question someone asked me later that day about Tabular Editor and which I’m definitely going to be asked about DAX Optimizer, namely:
This looks great, but it’s expensive and it’s hard for me to get sign-off to use third-party tools like this. Why doesn’t Microsoft give me something like this for free?
Before I carry on, let me make a few things clear:
- I work for Microsoft but these are my personal opinions.
- I have known many of the people involved in Tabular Editor and DAX Optimizer, including Marco and Alberto, for many years and have had business relationships with them in the past before working for Microsoft.
- I don’t endorse any non-Microsoft Power BI-related commercial tools here on my blog but I do use many of them and mention them regularly, leaving readers to draw their own conclusions. This post is not an endorsement of Tabular Editor or DAX Optimizer.
With that out of the way let me address some of the different aspects of this question.
There’s a post on the Power BI blog from 2021 here co-written by Marco Russo and Amir Netz which covers Microsoft’s official position on community and third party Power BI development tools and which is still relevant. There’s also a companion article by Marco here that’s worth reading. In summary Microsoft’s long-term goal is to provide great tools for all Power BI developers, including enterprise developers, but in the meantime our priority is to build a solid platform that other people can build these tools on. I know many of you won’t believe me but here at Microsoft we have finite development resources and we need to make difficult decisions about what we invest in all the time. We can’t build every feature that everyone wants immediately and everyone wants different features.
As a result there will always be space for free and commercial third-party tools to innovate in the Power BI ecosystem. In the same way Tabular Editor serves the enterprise tools market, the vendors in the Power BI custom visuals marketplace extend Power BI with custom visuals. There are literally hundreds of other examples I could give in different areas such as planning and budgeting and admin and governance. Why doesn’t Microsoft buy some or all of these tools? We do buy tools vendors sometimes, but I feel these tools and companies tend to fare better outside Microsoft where they can compete with each other and move quickly, and when there’s a vibrant partner ecosystem around a product then the customer is better off too.
DAX Optimizer is slightly different to Tabular Editor and these other tools though. While the tool is very sophisticated the tool itself is not the whole point; it’s like a much, much more sophisticated version of Tabular Editor’s Best Practices Analyzer feature, a feature which is available in both the free and paid versions of Tabular Editor. The real value lies in the IP inside DAX Optimizer: these aren’t just any rules, these are Marco and Alberto’s rules for optimising DAX. Anyone could build the tool, but only Marco and Alberto could write these particular rules. I guess that’s why the Tabular Editor team had these stickers on their stand on Friday:
Doesn’t Microsoft have people who are this good at DAX who could write the same rules? We do have people who know more about DAX than Marco and Alberto (namely the people who create it, for example Jeffrey Wang) and we do have people who are extremely good at performance tuning DAX (for example my colleagues Michael Kovalsky or Phil Seamark). Indeed, back in 2021 Michael Kovalsky published a free set of rules here which you can use with Best Practices Analyzer in Tabular Editor and which represent the Power BI CAT team’s best practices recommendations on DAX and modelling, so you can argue that Microsoft already does offer a free solution to the problem that DAX Optimizer is trying to solve.
Marco and Alberto are Marco and Alberto though. They have a very strong brand. Consultancy is a famously hard business to scale and this is a very clever way for them to scale the business of DAX performance tuning. If you want their help in whatever form then you’ll need to pay for it. Couldn’t Microsoft just hire Marco and Alberto? I doubt they’d say yes if we asked, and in any case the situation is the same as with buying the tools I mentioned above: I think they add more value to the Power BI ecosystem outside Microsoft than they ever could inside it.
I’ve been lucky enough to get an invitation code to test DAX Optimizer and will be doing so this week, but I deliberately wrote this post before giving it a try. It’s important for me to stay up-to-date with everything happening in the world of Power BI because the customers I work with ask for my opinion. I wish the team behind it well in the same way I wish anyone who tries to build a business on top of Power BI well; the more successful they are, the more successful Power BI and Fabric are.
I really wish there was some kernel of the Power BI data/formula engine that was open source and could be used as a foundation for other projects. I think the quality and quantity of community projects would vastly expand if this were to happen. I notice that the symbiotic “msmdsrv.exe” process already launches in the Power BI desktop, and in Excel, and Visual Studio …. But if the engine was ever open-sourced then you would start to see this launching as an embedded engine in TONS of other apps. (similar to how we see sqlite being used as an embedded engine everywhere).
The Microsoft tabular engine has been around for a long time, and I don’t think Microsoft would have that much to fear by releasing an open source variation that includes code which is already 5 or 10 years old. Lots of other companies have done similar things. Eg. “Spark” is an open source product, that was shared with the community. Even Microsoft has benefitted from all contributions of the Spark community in a lot of different places (eg. it is repackaged in Synapse and other services).
It is still a bit surprising to me that there are people in this community have invested so much time and energy on top of a proprietary, non-opensource platform. At any instant Microsoft may steer the underlying platform in a different direction, or stop supporting the components we rely on, or even develop some new stuff of their own that competes with community stuff, or potentially even send their lawyers after you. There are lots of risks, and relatively few rewards…
@Chris
“at Microsoft we have finite development resources and we need to make difficult decisions about what we invest in all the time. ”
After all, with products like Cortona, Sway, Delve, etc to focus on and the constant need to man teams responsible for changing names, logos and themes – I have no doubt there are finite resources at M$ for fringe products like Excel, Power BI etc. especially when the Community is doing such a great job at filling in the gaps.
Cheers
Sam
Interesting post Chris and refreshing to see a Microsoft employee discussing this topic openly and with a healthy dose of pragmatism.
@Sam
Microsoft is a very large company and product strategy/branding/marketing managers take many of the decisions you are referring to. The engineering people may or may not agree with / be able to influence those decisions – much the same as in many very large companies, particularly when the products you refer to are some distance away in the organisation. The engineers and their programme managers work within the constraints they have and take the best decisions they can (i.e. prioritise).
Also worth noting – many current Microsoft employees have worked with Microsoft products for a long time both inside and outside of Microsoft, so they do see the bigger/whole picture.
In response to the paragraph below the picture, while it is good that Microsoft has employees that are better than Marco and Alberto at DAX, it doesn’t do us much good if they don’t share much of what they know. All 3 of the blog posts that you have linked don’t have much recent content. I would be interested to learn from the Microsoft employees any tips on how to optimize DAX beyond what SQLBI covers in their Optimizing DAX 2nd Edition course (great course, BTW).
In response to the 2nd paragraph below the picture, there is quite a bit of guessing and assuming and almost putting words into Marco and Alberto’s mouths. Has Microsoft tried offering them jobs or a different way for them to partner with Microsoft? The results might be better than you expect!
I don’t think we have anyone, apart from the developers themselves, who know more about DAX optimisation than Marco and Alberto – although we do have some very good people. Marco and Alberto talk to the developers a lot (how do you think they know what they know…?) and I don’t think there is a vast amount of optimisation knowledge that is unshared at this point. Would this knowledge transfer happen more efficiently if Marco and Alberto were Microsoft employees? No – I think we partner with them very effectively, and I don’t think they or we would want to change that relationship.