Apache Kafka Series - Kafka Connect Hands-on Learning (Learn Apache Kafka Connect Framework with Examples)

Welcome! We have designed a completely new course, "Apache Kafka Series - Kafka Connect Hands-on Learning,” to help you understand the Apache Kafka Connect framework using a practical approach. People who know about the basics of Kafka and own a Windows, Linux, or Mac machine that is recent enough and has at least 4GB RAM and 5GB hard disk space are the best candidates for this particular course. Though it is not compulsory to have an understanding of Docker, knowledge of Linux command lines is advantageous. You may wish to keep a notebook and pen close by to jot down important points if you choose to take this tutorial.

Apache Kafka Connect is a powerful tool that makes it easy to stream data between Apache Kafka and other systems. It serves as a general framework for producers and consumers in Kafka through its API, Runtime, and REST services that help developers create connectors that move large volumes of data in real time. Like Apache Kafka, Kafka Connect extends itself with features like fault tolerance or elasticity, making it available on demand based on workloads, collecting metrics, ingesting entire databases, and gathering logs from application servers into up-to-date Kafka topics with low latency, enabling efficient stream processing.

In the course “Apache Kafka Series - Kafka Connect Hands-on Learning,” you will be taken through the process of deploying, configuring, and managing Kafka Connect USING practical in-class exercises. This will enable your organization to scale the service either for a bigger firm or for a small product environment, as it has both distributed and standalone modes. You will also discover how to use the REST interface that is used for submitting and controlling connectors on your Kafka connect cluster. Mastering Apache Kafka Connect can greatly enhance your career because there are not many professionals available who know Kafka.

Apache Kafka Series - Kafka Connect Hands-on Learning Table of Contents:

1. Important Pre-Requisites (01:09)
2. Course Objectives (01:20)
3. Course Structure (01:55)
4. About Your Instructor (02:45)
5. Code Download (00:04)
6. What is Kafka Connect? (04:50)
7. Kafka Connect Architecture Design (03:20)
8. Connectors, Configuration, Tasks, Workers (03:03)
9. Standalone vs Distributed Mode (02:11)
10. Distributed Architecture in Detail (02:58)
11. Important Information about Installation (01:01)
12. Docker on Mac (Recent Versions) (03:54)
13. Docker Toolbox on Mac (Older Versions) (05:16)
14. Docker on Linux (Ubuntu Example) (04:56)
15. Docker on Windows 10 64bit (03:42)
16. Docker Toolbox on Windows (Older Versions) (05:37)
17. Important Note for Docker Toolbox Users (00:28)
18. Starting Kafka Connect Cluster Using Docker Compose (03:40)
19. Troubleshooting: It's Not Working! What to Do? (02:49)
20. Where to View Logs? (04:05)
21. Asking a Question in the Q&A (02:39)
22. Kafka Connect Source Architecture Design (01:14)
23. FileStream Source Connector - Standalone Mode - Part 1 (09:05)
24. FileStream Source Connector - Standalone Mode - Part 2 (07:56)
25. FileStream Source Connector - Distributed Mode (11:29)
26. List of Available Connectors (05:04)
27. Twitter Source Connector - Distributed Mode - Part 1 (07:37)
28. Twitter Source Connector - Distributed Mode - Part 2 (06:11)
29. Section Summary (01:36)
30. Kafka Connect Sink Architecture Design (01:43)
31. ElasticSearch Sink Connector - Distributed Mode - Part 1 (08:58)
32. ElasticSearch Sink Connector - Distributed Mode - Part 2 (07:31)
33. Kafka Connect REST API (11:08)
34. JDBC Sink Connector - Distributed Mode (13:06)
35. Goal of the Section: GitHubSourceConnector (01:34)
36. Finding the Code and Installing Required Software (03:09)
37. Description of the GitHub Issues API (03:55)
38. Using the Maven Archetype to Get Started (05:43)
39. Config Definitions (12:31)
40. Connector Class (06:57)
41. Writing a Schema (07:31)
42. Data Model for Our Objects (06:10)
43. Writing Our GitHub API HTTP Client (07:03)
44. Source Partition & Source Offsets (04:22)
45. Source Task (13:40)
46. Building and Running a Connector in Standalone Mode (04:59)
47. Command for Next Lecture (00:20)
48. Deploying Our Connector on the Landoop Cluster (06:30)
49. More Resources for Developers (01:57)
50. Setting up Kafka Connect in Production (Part 1) (14:19)
51. Setting up Kafka Connect in Production (Part 2) (06:15)
52. What's Next? (01:46)
53. Congratulations and Thank You! (01:18)
54. Thank You! (01:32)
55. Bonus Lecture (00:53)

Who is this course for?

1. Developers interested in learning about the Apache Kafka Connect Framework.
2. Professionals knowledgeable about the Apache Kafka ecosystem and basic core concepts
3. Architects looking to understand how Kafka Connect fits into their Solution Architecture

Click on the links below to Download Apache Kafka Series - Kafka Connect Hands-on Learning!