Skip to content
Merged
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -23,13 +23,13 @@ title: Deployment options
A unique thing about Samza is that it provides multiple ways to run your applications. Each deployment model comes with its own benefits, so you have flexibility in choosing whichever fits your needs. Since Samza supports “Write Once, Run Anywhere”, your application logic does not change depending on where you deploy it.

### Running Samza on YARN
Samza integrates with [Apache YARN](learn/documentation/{{site.version}}/deployment/yarn.html) for running stream-processing as a managed service. We leverage YARN for isolation, multi-tenancy, resource-management and deployment for your applications. In this mode, you write your Samza application and submit it to be scheduled on a YARN cluster. You also specify its resource requirement - the number of containers needed, number of cores and memory required per-container. Samza then works with YARN to provision resources for your application and run it across a cluster of machines. It also handles failures of individual instances and automatically restarts them.
Samza integrates with [Apache YARN](/learn/documentation/{{site.version}}/deployment/yarn.html) for running stream-processing as a managed service. We leverage YARN for isolation, multi-tenancy, resource-management and deployment for your applications. In this mode, you write your Samza application and submit it to be scheduled on a YARN cluster. You also specify its resource requirement - the number of containers needed, number of cores and memory required per-container. Samza then works with YARN to provision resources for your application and run it across a cluster of machines. It also handles failures of individual instances and automatically restarts them.

When multiple applications share the same YARN cluster, they need to be isolated from each other. For this purpose, Samza works with YARN to enforce cpu and memory limits. Any application that uses more than its requested share of memory or cpu is automatically terminated - thereby, allowing multi-tenancy. Just like you would for any YARN-based application, you can use YARN's [web UI](/learn/documentation/{{site.version}}/deployment/yarn.html#application-master-ui) to manage your Samza jobs, view their logs etc.

### Running Samza in standalone mode

Often you want to embed and integrate Samza as a component within a larger application. To enable this, Samza supports a [standalone mode](learn/documentation/{{site.version}}/deployment/standalone.html) of deployment allowing greater control over your application's life-cycle. In this model, Samza can be used just like any library you import within your Java application. This is identical to using a [high-level Kafka consumer](https://kafka.apache.org/) to process your streams.
Often you want to embed and integrate Samza as a component within a larger application. To enable this, Samza supports a [standalone mode](/learn/documentation/{{site.version}}/deployment/standalone.html) of deployment allowing greater control over your application's life-cycle. In this model, Samza can be used just like any library you import within your Java application. This is identical to using a [high-level Kafka consumer](https://kafka.apache.org/) to process your streams.

You can increase your application's capacity by spinning up multiple instances. These instances will then dynamically coordinate with each other and distribute work among themselves. If an instance fails for some reason, the tasks running on it will be re-assigned to the remaining ones. By default, Samza uses [Zookeeper](https://zookeeper.apache.org/) for coordination across individual instances. The coordination logic by itself is pluggable and hence, can integrate with other frameworks.

Expand Down