Cloud Pub/Sub Google Professional Data Engineer GCP

In this, we will learn about Cloud Pub/Sub.

Cloud Pub/Sub:

Services can interact asynchronously using Pub/Sub, with latencies on the order of 100 milliseconds. In order to ingest and disseminate data, Pub/Sub utilizes streaming analytics and data integration pipelines. It works well as a messaging-oriented middleware for service integration as well as a queue for job parallelization. You may use Pub/Sub to establish systems of event producers and consumers, which refer to as publishers and subscribers. Publishers communicate with subscribers asynchronously by broadcasting events, rather than by synchronous remote procedure calls (RPCs). Further,

• Publish/subscribe messaging, or pub/sub messaging
• a form of asynchronous service-to-service communication
• used in serverless and microservices architectures.
• any message published to a topic is immediately received by all of the subscribers to the topic.
• Google Cloud Pub/Sub gives durable message ingestion service.
• Messages gets saved for 7 days
• Messages are opaque and its not processed by pubsub. We can send also metadata(key value pairs) to pubsub
• Project for the topic can be different from the project for subscription
• Subscriber has to ack within the timeout period
• By creating topics for streams or channels, enable different components of application to subscribe to specific streams of data without needing to construct subscriber-specific channels on each device.
• It also connects to GCP services to connect ingestion, data pipelines, and storage systems.
• Able to handle data spikes

Streaming Data Challenges

• Reliance on centralized storage and compute clusters.
• Centralized ownership
• Scalability bottlenecks
• Controlling network experience
• Business integration hiccups
• Streaming Data is Very Complex – continuously generate by an array of sources and devices in a wide variety of formats.

Pass the GCP Exam Now!