Ajit Monteiro, CTO & Co-Founder
In 2023, “Artificial Intelligence (AI)” transitioned from a mere buzzword to a tangible reality. Snowflake’s focus on Apps and AI led to strategic acquisitions and significant product updates, while Matillion introduced its cloud-native Data Productivity Cloud and Power BI introduced new features like CoPilot and DAX Query View. Explore how these advancements are transforming data analytics and intelligence.
This year, Snowflake made several acquisitions to improve their AI, data sharing, migration, and python offerings.
Myst AI for its time series forecasting abilities; crucial for optimizing business operations like supply chain management and financial planning
SnowConvert to bolster Snowflake’s migration toolkit, significantly easing the transition of legacy databases to Snowflake’s cloud data platform
LeapYear to incorporate differential privacy into Snowflake’s services, enabling secure data collaboration with mathematically proven privacy protection
Ponder to enrich Snowflake’s data capabilities with Ponder’s expertise in bridging data science libraries and scalable data operations, fostering enhanced analytics and machine learning functionalities
Neeva to integrate advanced, privacy-focused search capabilities into Snowflake’s data cloud, enhancing user experience and competitive edge in data analytics
Samooha to accelerate our vision for removing the technical, collaboration, and financial barriers to unlocking value with data clean rooms
(Generally Available): Snowflake has notably enhanced its Python support, allowing data scientists and engineers to integrate Python tools and libraries with Snowflake’s platform seamlessly. This enhancement facilitates efficient data analysis and machine learning directly within Snowflake, leveraging Python’s widespread popularity in the data science community to maximize the platform’s data analytics capabilities.
Why it matters: Allows you to use Python inside of Snowflake, including in user-defined functions, and workbooks.
(Available on AWS, Private Preview on Azure): Snowflake has expanded its ecosystem by introducing native applications, a move that significantly enriches the functionality and flexibility of its cloud data platform. These native apps, developed either by Snowflake or third-party developers, are designed to operate seamlessly within Snowflake’s environment, offering users a range of tools and services directly accessible within the platform.
Why it matters: Allows for developing and publishing data applications inside of Snowflake.
(Open Preview): Dynamic Tables allow users to define transformation logic with a simple, declarative SELECT statements, and Snowflake will automatically keep the table up-to-date, with cost-efficient incremental updates, a custom-defined refresh schedule.
Why it matters: Dynamic tables removes the need for some customers to maintain their own incremental update ETL framework.
(Open Preview): You can now maintain your entire database schema in git and deploy automatically via the Snowpark CLI. Snowflake’s new CREATE OR ALTER statement will handle DDL modifications automatically without dropping objects entirely. Check out our blog post to learn more.
Why it matters: Allows development teams to work simultaneously on a codebase while tracking and managing changes efficiently.
(Generally Available): The new SYSTEM$SEND_EMAIL() system stored procedure can be used to send- email notifications. You can call this stored procedure to send an email notification from a task, your own stored procedure, or an interactive session.
Why it matters: Allows for data-driven email notifications for use-cases like KPI thresholds, data observability, access monitoring, etc.
(Open Preview): You can create secure access to specific network locations external to Snowflake, then use that access from within the handler code for user-defined functions (UDFs) and stored procedures. You can enable this access through an external access integration.
Why it matters: Allow or block specific network access to Snowflake, as well as call external APIs from Snowflake.
(Expected Release in Summer 2024): Enables developers to deploy, manage, and scale generative AI and full-stack apps, including infrastructure options like GPUs, securely within Snowflake. This service broadens access to third-party services like LLMs, Notebooks, MLOps tools, and more.
Why it matters: Allows you to host code from any language inside of Snowflake, and make sure your data that is used by the code doesn’t leave the Snowflake cloud.
(Expected Release in Summer 2024): A new LLM developed by Snowflake, built from the acquisition of Applica’s generative AI technology, helps customers extract insights from unstructured data without needing machine learning expertise
Why it matters: Easy access to document-based machine learning.
(Expected Release in Summer 2024): A new LLM developed by Snowflake, built from the acquisition of Applica’s generative AI technology, helps customers extract insights from unstructured data without needing machine learning expertise
Matillion released its Data Productivity Cloud product, which is a cloud-native version of its software.
Why it matters: In addition to enhanced functionality, moving to the SaaS version of Matillion will remove the need to host and manage their own instance, allowing more time to focus on business value.
In Q1 2024, Matillion expects to launch an AI Prompt component, allowing users to augment their data pipelines with GenAI.
Why it matters: Companies with an existing Matillion implementation may find this feature to be an easy route to proof-of-concept, before investing heavily in a custom LLM integration.
Announced in July 2023, PipelineOS is the key innovation in Matillion’s new Data Productivity Cloud. The stateless, microservice architecture allows for both horizontal and vertical scaling.
Why it matters: Understanding the concept of PipelineOS will better equip customers to transition to the SaaS platform.
Power BI continues to redefine the landscape of data visualization with its new features, including CoPilot and the DAX Query View. Additionally, Microsoft released new branding for all of their data-related tools: Microsoft Fabric.
Using Copilot, you can simply describe the visuals and insights you’re looking for, and Copilot will do the rest. Users can create and tailor reports in seconds, generate and edit DAX calculations, create narrative summaries, and ask questions about their data, all in conversational language.
Why it matters: Allows you to create dashboards using natural language, minimizing the need for a analytics engineer, especially for ad-hoc use cases.
A new feature that provides powerful querying capabilities, enhancing data analysis and insights derivation.
Why it matters: Gives you access to writing DAX queries inside of PowerBI without any external tools.
Announced in May 2023, Microsoft Fabic is the new branding for all data-related tools in the Microsoft ecosystem (e.g. Data Factory, PowerBI, Synapse, etc.).
Why it matters: Currently, Snowflake and Databricks are considered to be the industry leading data cloud platforms, and Fabric is Microsoft’s response to those tools.
In the ever-evolving landscape of business intelligence, several key trends are reshaping how organizations perceive, manage, and leverage their data assets. From the imperative of AI-ready data to the paradigm shift of treating data as a product, these trends encapsulate transformative approaches driving the future of business analytics. As businesses head into 2024, understanding and adapting to these trends is paramount for sustained growth and innovation. Let’s delve into each trend, uncovering their significance and impact for modern data organizations.
If your data isn’t ready for AI, then your organization isn’t ready for AI. Before jumping on the AI bandwagon, companies need to take a step back and assess their data landscape. An AI algorithm’s effectiveness is tied to the quality of data it processes.
While 79% of executives plan to boost investments in AI/ML, a mere 24% consider themselves truly data-driven (Wavestone). As Forrester aptly notes, while AI may be ready for the spotlight, it’s the quality of data and analytics that truly determine if it shines. Enhancing data quality isn’t just a peripheral task; it’s a strategic move that can boost AI/ML model accuracy by a significant 20% (Forrester). So what exactly does it mean for data to be AI-ready? According to Gartner, for data to meet the AI-ready criteria, it must be secure, enriched, fair, accurate, and governed.
To help your organization navigate the complexities of preparing your data for AI, OneSix has built some free tools and resources:
In the pursuit of unified business visibility, data analytics teams encounter a significant hurdle in finding data within the organization, with a notable 24% citing this challenge as their foremost struggle in executing data strategies during 2023 (Forrester). To address this, Generative AI will increase the trend toward centralized data that creates a single source of truth for large language models (LLMs) and everything else (Snowflake). This shift aligns with the evolving need for a consolidated and authoritative data repository to streamline organizational data.
The explosion of unstructured and semi-structured data further underscores the necessity for organizations to embrace all-in-one unified data platforms. Forrester highlights that these platforms are essential for effective cost management, support for diverse multi-structured data analytics, and enabling broader use cases and workloads.Embedded analytics is going to play a pivotal role in building a data-driven culture. By integrating analytics capabilities and data visualizations directly into user workflows, applications, or portals, embedded analytics streamlines access to insights and provides users with a highly interactive and user-friendly data engagement experience. OneSix can help you get there, leveraging BI platforms like Power BI, Sigma, Tableau, Sisense, and Pyramid Analytics. Get in touch with us to learn more.
The evolving market trend emphasizes a transformative shift in how data is perceived— treating data as a product. This involves ensuring datasets are discoverable, addressable, self-describing, interoperable, trustworthy, and secure. The consumers of these datasets can then be other departments and organizations through data sharing technologies. This shift presents an opportunity for organizations to enrich their internal repositories by augmenting them with externally published datasets.
These external sources, such as economic indicators, population statistics, or weather data, offer a chance to enhance the depth and breadth of insights, empowering businesses with a more comprehensive and diverse informational landscape to drive informed decision-making and innovation.
Vector databases are rapidly emerging as a highly efficient storage solution tailored for the demands of AI, big data applications, and the long-term memory requirements of Large Language Model use cases. They excel at similarity-based searches, proving invaluable for critical functions like image recognition, recommendation systems, and intricate data comparisons.
What sets vector databases apart is their remarkable scalability and rapid data processing capabilities, making them ideal for real-time operations and large-scale applications. Their adaptability and speed render them not just as a viable option but as a crucial asset in the evolving landscape of data storage and utilization.
As we head into 2024, the world of data and AI technology is poised for significant transformation. The trends witnessed in 2023—AI-ready data, unified business visibility, cloud-first approaches, data-as-a-product paradigm, and real-time analytics—carry profound implications for businesses across all industries. Understanding and jumping into these trends is a must for any business aiming to stay ahead in this fast-moving world of data and AI.
