- Real-Time Data Streaming on Cloud Platforms: Leveraging Cloud Features for Real-Time Insights
Editor's Note: The following is an article written for and published in DZone's 2024 Trend Report, Data Engineering: Enriching Data Pipelines, Expanding AI, and Expediting Analytics.Businesses today rely significantly on data to drive customer eng…
- 8 days ago 6 Nov 24, 12:00pm - - Optimizing Your Data Pipeline: Choosing the Right Approach for Efficient Data Handling and Transformation Through ETL and ELT
Editor's Note: The following is an article written for and published in DZone's 2024 Trend Report, Data Engineering: Enriching Data Pipelines, Expanding AI, and Expediting Analytics.As businesses collect more data than ever before, the ability to…
- 9 days ago 5 Nov 24, 12:00pm - - The Modern Era of Data Orchestration: From Data Fragmentation to Collaboration
Editor's Note: The following is an article written for and published in DZone's 2024 Trend Report, Data Engineering: Enriching Data Pipelines, Expanding AI, and Expediting Analytics.Data engineering and software engineering have long been at odds,…
- 10 days ago 4 Nov 24, 8:00pm - - Digitalization of Airport and Airlines With IoT and Data Streaming Using Kafka and Flink
The digitalization of airports faces challenges such as integrating diverse legacy systems, ensuring cybersecurity, and managing the vast amounts of data generated in real time. The vision for a digitalized airport includes seamless passenger experie…
- 10 days ago 4 Nov 24, 2:00pm - - Optimizing Vector Search Performance With Elasticsearch
In an era characterized by an exponential increase in data generation, organizations must effectively leverage this wealth of information to maintain their competitive edge. Efficiently searching and analyzing customer data — such as identifying us…
- 10 days ago 4 Nov 24, 1:00pm - - Data Governance Essentials: Glossaries, Catalogs, and Lineage (Part 5)
What Is Data Governance, and How Do Glossaries, Catalogs, and Lineage Strengthen It?Data governance is a framework that is developed through the collaboration of individuals with various roles and responsibilities. This framework aims to establish p…
- 13 days ago 1 Nov 24, 3:00pm - - How to Identify Bottlenecks and Increase Copy Activity Throughput in Azure Data Factory
Azure Data Factory (ADF) is a cloud-native ETL tool to process data seamlessly across different sources and sinks.Copy activity is mostly used to copy data from one source to another source. While copying data between two different sources, we need…
- 14 days ago 31 Oct 24, 8:00pm - - Inside the World of Data Centers
The computing requirements of algorithms have increased dramatically over the past two decades. In particular, machine learning (ML) algorithms have experienced a growth in computing resource demand that exceeds Moore’s Law. While Moore's Law predi…
- 16 days ago 29 Oct 24, 9:00pm - - How to Design Event Streams, Part 1
Event streaming is becoming increasingly common in the world today. An event is a single piece of data that describes, as a snapshot in time, something important that happened in your business. We record that data to an event stream (typically using…
- 17 days ago 28 Oct 24, 3:00pm - - The Power of Market Disruption: How to Detect Fraud With Graph Data
In previous articles, I’ve mentioned my short career in the music industry. Let me tell a quick story about something really cool that happened while playing keyboards on a new artist project in 1986. Emerging from the solo section of the first s…
- 17 days ago 28 Oct 24, 11:00am - - Reactive Kafka With Spring Boot
Event-driven architectures are at the core of modern, scalable systems. Reactive Kafka, when combined with Spring Boot and WebFlux, offers a powerful approach to building non-blocking, high-throughput services. In this article, we’ll focus on build…
- 20 days ago 25 Oct 24, 1:30pm - - High-Speed Real-Time Streaming Data Processing
From data ingestion to reporting, the primary goal is to convert data into actionable information. Online data is growing at a much faster rate than data processing speeds. For businesses to stay competitive, data must be readily available for maki…
- 21 days ago 24 Oct 24, 4:00pm - - Minimizing Latency in Kafka Streaming Applications That Use External API or Database Calls
Kafka is widely adopted for building real-time streaming applications due to its fault tolerance, scalability, and ability to process large volumes of data. However, in general, Kafka streaming consumers work best only in an environment where they do…
- 22 days ago 23 Oct 24, 4:00pm - - Leveraging Event-Driven Data Mesh Architecture With AWS for Modern Data Challenges
In today's data-driven world, businesses must adapt to rapid changes in how data is managed, analyzed, and utilized. Traditional centralized systems and monolithic architectures, while historically sufficient, are no longer adequate to meet the growi…
- 22 days ago 23 Oct 24, 1:00pm - - Building Predictive Analytics for Loan Approvals
In this short article, we'll explore loan approvals using a variety of tools and techniques. We'll begin by analyzing loan data and applying Logistic Regression to predict loan outcomes. Building on this, we'll integrate BERT for Natural Language Pro…
- 22 days ago 23 Oct 24, 12:00pm - - Automate Private Azure Databricks Unity Catalog Creation
Disclaimer: All the views and opinions expressed in the blog belong solely to the author and not necessarily to the author's employer or any other group or individual. This article is not a promotion for any cloud/data management platform. All the i…
- 24 days ago 21 Oct 24, 5:00pm - - Tech Trends 2024: Highlights on the Current Tech Industry From a Developer
I have attended several events this year, and I’m constantly keeping my ear to the ground for the latest topics and trends in technology. As a developer focused mostly on data and database industries, I feel that this year has seen a massive expans…
- 24 days ago 21 Oct 24, 3:00pm - - The Battle of Data: Statistics vs Machine Learning
The goal of this article is to investigate the fields of statistics and machine learning and look at the differences, similarities, usage, and ways of analyzing data in these two branches. Both branches of science allow interpreting data, however, th…
- 31 days ago 14 Oct 24, 8:00pm - - Hello, K.AI: How I Trained a Chatbot of Myself Without Coding
Generative AI (GenAI) enables many new use cases for enterprises and private citizens. While I work on real-time enterprise-scale AI/ML deployments with data streaming, big data analytics, and cloud-native software applications in my daily business l…
- 31 days ago 14 Oct 24, 6:00pm - - Optimizing IoT Performance in Industrial Environments
Internet of Things (IoT) devices have become common in industrial environments, giving users better visibility, control, and capabilities. However, making the IoT product work well requires knowing how to optimize software and hardware-related aspect…
- 36 days ago 9 Oct 24, 12:00pm -