ReinforcementLearningEnvironments.jl

⚠️ This package is moved into ReinforcementLearning.jl (2021-05-06)

This package serves as a one-stop place for different kinds of reinforcement learning environments.

Install:

pkg> add ReinforcementLearningEnvironments

API

All the environments here are supposed to have implemented the AbstractEnv related interfaces in ReinforcementLearningBase.jl.

Supported Environments

By default, only some basic environments are installed and exported to make sure that they work on different kinds of platforms. If you want to use some other environments, you'll need to add those dependencies correspondingly.

Built-in Environments

Traits		1	2	3	4	5	6	7	8	9	10	11	12	13
ActionStyle	MinimalActionSet	✔	✔	✔		✔		✔	✔	✔	✔	✔	✔	✔
	FullActionSet				✔		✔
ChanceStyle	Stochastic	✔		✔	✔						✔	✔	✔	✔
	Deterministic		✔			✔	✔
	ExplicitStochastic							✔	✔	✔
DefaultStateStyle	Observation	✔	✔	✔	✔	✔	✔		✔		✔	✔	✔	✔
	InformationSet							✔		✔
DynamicStyle	Simultaneous					✔
	Sequential	✔	✔	✔	✔		✔	✔	✔	✔	✔	✔	✔	✔
InformationStyle	PerfectInformation		✔				✔		✔
	ImperfectInformation	✔		✔	✔	✔		✔		✔	✔	✔	✔	✔
NumAgentStyle	MultiAgent					✔	✔	✔	✔	✔
	SingleAgent	✔	✔	✔	✔						✔	✔	✔	✔
RewardStyle	TerminalReward	✔	✔		✔	✔	✔	✔	✔	✔
	StepReward			✔							✔	✔	✔	✔
StateStyle	Observation	✔	✔	✔	✔	✔	✔		✔		✔	✔	✔	✔
	InformationSet							✔		✔
	InternalState			✔
UtilityStyle	GeneralSum	✔	✔	✔	✔						✔	✔	✔	✔
	ZeroSum					✔	✔			✔
	ConstantSum								✔
	IdenticalUtility							✔

1. MultiArmBanditsEnv
2. RandomWalk1D
3. TigerProblemEnv
4. MontyHallEnv
5. RockPaperScissorsEnv
6. TicTacToeEnv
7. TinyHanabiEnv
8. PigEnv
9. KuhnPokerEnv
10. AcrobotEnv
11. CartPoleEnv
12. MountainCarEnv
13. PendulumEnv

Note: Many traits are borrowed from OpenSpiel.

3-rd Party Environments

Environment Name	Dependent Package Name	Description
AtariEnv	ArcadeLearningEnvironment.jl
GymEnv	PyCall.jl
OpenSpielEnv	OpenSpiel.jl
SnakeGameEnv	SnakeGames.jl	SingleAgent/Multi-Agent, FullActionSet/MinimalActionSet
#list-of-environments	GridWorlds.jl	Environments in this package use the interfaces defined in RLBae directly

Usage

julia> using ReinforcementLearningEnvironments

julia> using ReinforcementLearningBase

julia> env = CartPoleEnv()
# CartPoleEnv

## Traits

| Trait Type        |                  Value |
|:----------------- | ----------------------:|
| NumAgentStyle     |          SingleAgent() |
| DynamicStyle      |           Sequential() |
| InformationStyle  | ImperfectInformation() |
| ChanceStyle       |           Stochastic() |
| RewardStyle       |           StepReward() |
| UtilityStyle      |           GeneralSum() |
| ActionStyle       |     MinimalActionSet() |
| StateStyle        |     Observation{Any}() |
| DefaultStateStyle |     Observation{Any}() |

## Is Environment Terminated?

No

## State Space

`Space{Array{IntervalSets.Interval{:closed,:closed,Float64},1}}(IntervalSets.Interval{:closed,:closed,Float64}[-4.8..4.8, -1.0e38..1.0e38, -0.41887902047863906..0.41887902047863906, -1.0e38..1.0e38])`

## Action Space

`Base.OneTo(2)`

## Current State

[-0.032343893118127506, -0.04221525994544837, 0.024350079684957393, 0.04059943022508135]

julia> state(env)
4-element Array{Float64,1}:
  0.02688439956517477
 -0.0003235577964125977
  0.019563124862911535
 -0.01897808522860225

julia> action_space(env)
Base.OneTo(2)

julia> while !is_terminated(env)
           env(rand(legal_action_space(A)))
       end

julia> using ArcadeLearningEnvironment  # to use 3rd-party environments, you need to manually install and use the dependency first

julia> env = AtariEnv("pong");

Environment Wrappers

Many handy environment wrappers are provided to mimic the OOP style manipulation.

• ActionTransformedEnv
• DefaultStateStyleEnv
• MaxTimeoutEnv
• RewardOverriddenEnv
• StateCachedEnv
• StateOverriddenEnv

Application

Checkout experiments in ReinforcementLearningZoo.jl for how to apply modern reinforcement learning algorithms to these environments. You may also want to read this pluto notebook for how to write a customized environment.