MS Ignite Day #2 Recap

Day 2 was my opportunity to be focused and dive into a few technologies that were really peaking my interest, namely Azure Synapse Analytics and SQL Server 2019. Since I heard the Synapse announcement I wanted to make sure I sought out every session on this and understood everything I could possible learn and today certainly went a long way to help me understanding this new technology and also realized there is so much more to learn, but there are still 3 more days left.

With Azure Synapse everything I keep learning about this is it’s certainly impressive and I will stand by the statement that Snowflake, Google Big Query and Amazon Redshift better be paying attention because the complete integrated story, performance benchmarks and cost model have closed some big gaps that previously existed with Azure Data Lake and Azure SQL DW. One of the sessions I had a chance to attend was “Democratizing the Data Lake with On-Demand Capabilities in Azure Synapse Analytics” with Josep Aguilar Saborit and Charles Feddersen, where I came away with two very important takeaways. First, was the ease of being able to query and import data, in a nutshell get the data into Azure Data Lake Storage and it’s available. With that said it seems that Microsoft and most of the industry are standardizing on parquet as the file format of choice. Which if you are not familiar with is an extremely easy file format to get almost any file into, Azure Data Factory can really help here. Parquet format files are first class citizens within Azure Synapse and querying or importing this data is almost effortless.

Now let’s talk a little bit about the architecture behind Synapse and while yes Azure Synapse can be thought of as Azure SQL DW 2.0 you need to take everything you may already know about Azure SQL DW architecture and throw it out the window. The most important thing to remember is that the architecture behind this new service has been reinvented from the ground up and made to be a cloud first design. If you remember Azure SQL DW was not a cloud first design as it was basically the predecessor of SQL Server PDW just ported into the cloud with very little architecture change.

Now let me caveat the next few statements with I still have a lot to learn about Azure Synapse architecture but here are a few items that I have picked up on and I have been very impressed by. Both the failure detection and hotspot detection that is baked into the solution is quite impressive. First let’s understand that queries are natively treated as distributed and alight with both Hash and User partitions with the execution nodes, see picture below (Sorry these are pictures taken during presentations)

With failure detection being handled at the node level it allows for any failure at a node level to be automatically recovered and redistributed to the other active nodes. That’s right unless for some reason you lose all nodes your queries will not fail. If you are familiar with more legacy architectures, you would then know that this was not always the case.

Now with hot spot detection imagine the scenario that a query task was partitioned in a way where one node was doing a skewed amount of the work. What Synapse will do is recognize this skewed load and automatically redistribute this load to other nodes that are currently being underutilized. Oh, wait I am not done with the amazing special sauce, if multiple nodes become overloaded and distribution can’t be done then additional nodes can just be added to achieve ultimate performance.

Now let’s move on to SQL Server 2019 and new features in Azure SQL which I am just going to lump all in together. Now this is not a full comprehensive list of all new features just some things I wanted to call out. Let’s start with Azure SQL Serverless, if you are looking for a way to use Azure SQL and not incur a crazy cost then take a look at Azure SQL Serverless for scenarios such as Dev/Test environments (I’m sure there are other use cases but this is what I am focusing on for now). Typically in Dev/Test environments you don’t have the requirement to be up 100% of the time and therefore you don’t want to have the cost of running a database 24/7, and this is where serverless comes into play as you are only charged for the time that you are actually running queries. I will go out on a limb and say this could easily be a 40-50% cost savings for existing Dev/Test environments you have that are running on current Azure SQL Database and incurring 24/7 costs.

One more feature set I want to quickly hit on is the Intelligent Query Processing. With this new feature there are quite a number of areas of improvement so let’s just hit on a quick few. The first is the memory grant feedback, where if the initial execution if the memory grant was too small and caused spill to disk the subsequent executions will be adjusted to properly set memory grants and therefore reduce spill and increase query execution time. Table Variable deferred compilation now treats a table variable just as a temp table has always been done and therefore query cardinality is now no longer assumed to be a rowcount of 1. While I am not a huge table variable fan if you do have a workload that is very table variable heavy then just upgrading to SQL Server 2019 should result in performance gains just by upgrading.

One last note is on Azure Data Studio as I had a chance to see Alan Yu’s session on Azure Data Studio focused on notebooks. If you are not using Azure Data Studio and notebooks hurry up and start learning you will thank me later. Plus, Azure Data Studio now has PowerShell notebooks and is a one stop show for almost any database developer. Azure Data Studio November release

MS Ignite Day#1 Recap

 

What an amazing first day at Microsoft Ignite that all started at about 6:30 this morning when I got in line to be one of the first 3000 that got the opportunity to see the Keynote from Satya Nadella. There were so many new announcements made during the Keynote and I am sure that I am going to miss a few in my round up but will try to get all the ones I can remember and my own take on their impact. So here we go!!

 

First it was obvious that the theme of the keynote was, this is the year of the evolution of the hybrid cloud, where we will start seeing the growth, adoption and seamless integration of cloud services being to be deployed to Edge devices or even on your own datacenter hardware while all still being managed through Azure portal. Sticking to this theme let’s first talk about all the announcements made that are directly related to this:

I am going to focus here in what I think is the most significant game changer and that is Azure Arc. With the ability to now deploy Azure based services to any infrastructure with a Kubernetes endpoint , including AWS and Google cloud , will transform how people now think about cloud architecture and deployments. All of this while still managing those deployment from a single interface using Azure Portal or automation through the Azure API. For myself this now enables me to develop solutions that are truly hybrid and brings applications and data sources into a hybrid model, where previously I was stuck in an on-premise mode due to many different factors. From a database perspective organizations have sensitive data that due to security concerns, regulatory requirements or latency SLA’s migrating to the cloud is a near impossibility now with Azure Arc we can put still have them on premise in our data center while taking advantage of features such as hyperscale, updates and security monitoring. Since I could keep going on and on about this feature I will stop here for now and just end with “It’s a game changing announcement”

 

Next let’s quickly review some of the Azure Data Services announcements:

 

While all these new announcements are amazing in the own right I want to take a few minutes to call out the Azure Synapse Analytics, think of this as Azure DW rearchitected. It has many similarities and traits that were started with Azure SQL DW and it’s predecessor of SQL Server PDW, however the similarities stop there as Azure Synapse Analytics was re-architected from the ground up to be a true cloud based platform. If you used Azure DW one of the issues was getting the performance you desired at a reasonable cost and why competitors such as Snowflake and Redshift where taking a majority of the market share. While demos always highlight just how fast a system can be, who would put a demo together to show how slow a platform is, the performance and cost differences now with Azure Synapse Analytics change the entire conversation around what the future of your data warehouse should look like and now have capabilities such as automatic data ingestion that didn’t even exit before. One more quick note on this is that Azure Databricks also already has connectors for Azure Synapse Analytics, opening up so many more possibilities for data ingestion and machine learning.

 

Lastly, I will wrap up with the last area I wanted to highlight which was Power Apps. There were several announcements here such as Power Virtual Agents, Integration with Microsoft Teams and increased data sizes for Power BI. This is an area that I have not paid much attention to over the last year or so and most of the new capabilities were release a couple weeks ago with Release Wave 2. What really opened up my eyes here was all the capabilities that Power Apps now has and if you have not looked at this recently then you really should be. The integrated security and controls really allow you to extend capability to Business Users while not creating a culture of Shadow IT as it’s still a centrally managed platform where you can prevent data sprawl and unauthorized data usage. I’m seriously looking at this as a way to eliminate custom SharePoint development going forward.

 

In a nutshell Day 1 at Ignite was a great day and boy did my brain hurt at the end of the day as I was thinking of all the new possibilities of a true hybrid cloud architecture.

 

 

Viewpoints from CES 2016

While I never really enjoy a cross country flight back home my feet are very happy that I am finally sitting down for a few hours.  CES 2016 was once again another great year and has proven to me again why it’s a must attend conference that spans across many technologies and reaches almost every industry.  This was my 5th time attending and over those years I have represented technology in several different industries, but this was my first focusing on marketing.  In most of my previous years I was looking at what technology could be built on top of or integrated into what I was already doing.  However, this year the focus was more on different ways to interact with the consumer, along with understanding the consumers buying patterns across all industries.

There certainly was one very common theme that you saw at every turn, “Mobile is king”, and it’s not just the phone that we all can’t live without anymore.  It was also the theme in all the auto industries connected cars with Wi-Fi enabled cars, integrated entertainment with Amazon, to self driving vehicles.  If the car can drive itself, that gives you more time to spend on your mobile devices or enjoying the entertainment systems.  From a content delivery aspect imagine not having to rely on radio and the spoken work to get the attention of those who are driving and delivering targeting advertising based on the location of the vehicle.  In my opinion we are not far away from this becoming a reality in the next couple of years.   One additional thought here as well is the car becoming even more integrated and say when the gas light comes on, it uses GPS and Wi-Fi to automatically search for the nearest gas station and give you directions.

From a phone aspect there was every gadget you could think of from phones longer battery life, larger screens, devices to give you additional battery life, protect your phone, boost your signal and protect your personal information.  All these tell me one thing and that the mobile phone has become a device that most people no longer know how to live without, myself included.  In almost every keynote, session, panel discussion or fireside chat I attended they all talked about the importance of or new ways to interact with the consumer, who spends over 14 hours a day with the phone no further than an arms length away and averages around 4 hours actively looking at the screen.   There is no doubt that the shift of marketing dollars and content will only increase in the digital space as mobile continues to gain not only additional eyeballs but also a greater share of recreational time compared to TV.

From a tradition TV marketing stand point everyone was in agreement that TV viewership is down and that is a trend that most likely is not going to change.  However, DISH network is certainly trying to make sure that not everyone walks away from there TV or cuts their cord.  There new DSTB is capable of 4K resolution and more importantly it is integrated with Sling, YouTube and Netflix.  Which points to a new trend of, if you can’t beat them join them.  Netflix has certainly disrupted the likes of both cable and satellite providers so why not build in Netflix search and try and keep them from cutting the cord completely.  It’s also a very smart move as it gets a greater amount of information about the consumer, their watching habits and increases the ability to provide addressable media. 

Overall the media landscape and consumer interactions continue to evolve as once again shown this year as CES and this upcoming year will show great strides in perfecting the interaction with the consumer.  None of the would be capable thought without the innovations taking place with consumer technology.   It may scare some about how much everyone will know about them but I for one will enjoy the day when I am not getting hit with ads, emails, calls, etc.. that have nothing to do with me or what I am looking to buy.

Be an Expert not an Ehhpert!!!

As we embark on a new year I decided to take a look back at my career within I.T. trying to figure out what was the turning point.  When did it go from a job to a career?  What advice could I give to others?  After thinking about this for a couple days I realized that everything changed when I started to care more about what I learned than the paycheck, and that moment for me was about 15 years ago.  Accountants-Sweat-Shop

I was working at the time for what could be called an I.T. sweatshop with 70+ hour weeks and all I cared about was getting the next paycheck and paying my bills.  Days were long and I wasn’t learning anything new just plugging along, writing code and it was time to start looking for a new job.  About two weeks later I had a new job offer, but it was a decrease in pay, and with a young family at home I had a lot to think about.  Time to make the tough decision of money or the opportunity to learn more, and I chose to learn more.  At the time it seemed like a small step but it turned out to be one of the best decisions I ever made and would also make the same decision again about 10 years later.

Right now also happens to be the time at I.Predictus when we are setting the goals for a new year and this years theme is appropriately, Play It Up.  As a leader of an organization there are many things your constantly worrying about,  but the most important to me is how do I set a good example for my team and how do I keep my team focused and energized.

So how does all of this have anything to do with being an Expert? and what the heck is an Ehhpert?

An ehhpert is the person that is always every questions with:images

  • Ehhh let me think about it.
  • Ehhh I’m not sure
  • Ehhh I don’t know

 

If this is you then stop doing it immediately and make today be the day that you become an expert.

Becoming an Expert and Playing It Up go hand in hand and this is what I am challenging my entire team to do this year. Additionally, I encourage everyone to also challenge themselves or your teams to do the same.  Surrounding yourself with intelligent people raises everyone up another level and causes a greater accountability with peers and a little healthy competition.

How do you become an expert?

  • Find a lane and don’t stop learning
  • Don’t spread yourself thin in multiple areas
  • Be confident and come across with conviction
  • Help your team and teach them what you know
  • Let everyone know you’re an expert
    • Blog
    • Present
    • Speak