Search Results

Apache Kafka® Go Quick Start

Contents

Getting Started

This quick start guide provides basic instructions for writing Apache Kafka applications in Go for TIBCO Cloud™ Messaging.

Building and running samples

Follow the steps below to build and run the Go samples. The Go samples are implemented as a Go module. Dependencies are defined in the included go.mod files.

The Kafka Go SDK is not supported on Windows

Linux and macOS

  • Unzip kafka-go-samples.zip
  • Change into module directory: cd kafka-go-samples/tcm/samples/kafka
  • Set GOBIN: export GOBIN=$PWD/bin
  • Generate vendor directory: go mod tidy then go mod vendor
  • Build and install binaries: go install tibco.com/tcm/samples/kafka...
  • Use -f option to specify the configuration file.
  • Run samples: $GOBIN/kafka_consumer -f <path-to>/tcm-config.yaml

Description of sample applications

Producer

The producer demonstrates connecting an Apache Kafka Go application to Cloud Messaging and publishing Kafka messages on a topic:

kafka_producer:
  -T string
    	topic to send on (default "demo_tcm")
  -c int
    	the number of messages to send before exiting (default -1)
  -f string
    	the path to a configuration yaml (default "tcm-config.yaml")
  -i duration
    	the duration between sends, e.g. 500ms, 1s, etc. (default 1s)
  -id string
    	client identifier (default "kafka_producer")
  -t duration
    	duration before timeout, e.g. 1h, 5m, 10s, etc.

Consumer

The consumer demonstrates connecting an Apache Kafka Go application to Cloud Messaging, creating a subscription, and receiving Kafka messages on a topic:

kafka_consumer:
  -G string
    	group id (default "KafkaConsumer")
  -T string
    	topic to send on (default "demo_tcm")
  -c int
    	the number of messages received before exiting (default -1)
  -f string
    	the path to a configuration yaml (default "tcm-config.yaml")
  -id string
    	client identifier (default "kafka_consumer")
  -latest
    	set initial offset position to 'latest'
  -t duration
    	duration before timeout, e.g. 500ms, 10s, etc.

Connecting to TIBCO Cloud Messaging

The client configuration file contains all the information client applications need to securely connect to TIBCO Cloud Messaging. Generate the client configuration file using the roles REST API or user interface. Generate as many configuration files as needed for each Role.

Note: TIBCO Cloud Messaging samples require a client configuration file to run.

Connection Example

conf := sarama.NewConfig()
conf.Metadata.Full = true
conf.Version = sarama.V2_0_0_0
conf.ClientID = clientID
conf.Net.TLS.Enable = !insecure
conf.Net.SASL.Enable = true
conf.Net.SASL.Version = sarama.SASLHandshakeV1
conf.Net.SASL.User = options.KafkaUsername
conf.Net.SASL.Password = options.KafkaPassword
conf.Consumer.Return.Errors = true
conf.Consumer.Offsets.Initial = offset

// we advise that the group ID contain the TCM subscription ID
groupID = strings.ReplaceAll(options.KafkaUsername, "/", ".") + "." + groupID
client, err := sarama.NewConsumerGroup([]string{options.KafkaBroker}, groupID, conf)