[epic] flaky tests

[thaJeztah]

This is a list of tests that have shown to be flakey on master. This list is collected from this gist; https://gist.github.com/tonistiigi/9e91cbb968dea83113d09ce5f9dbff79 (generated from logs since January; some of these may already have been resolved)

Status	Count	Test	Tracking-issue
✅ #37315	443	DockerDaemonSuite.TestDaemonNoSpaceLeftOnDeviceError	#37277
	130	DockerSwarmSuite.TestAPIServiceUpdatePort	#36501 / #30427 / #36743
	129	DockerSwarmSuite.TestSwarmPublishDuplicatePorts	#36501 / #30427 / #36743
✅ #36750	124	TestMount	#34321 note: 100% of failures is on rhel-74-selinux-overlay2-stable
	119	DockerSuite.TestRunSeccompUnconfinedCloneUserns	note: 100% of failures is on centos-7-noselinux-devicemapper
	119	DockerSuite.TestRunSeccompAllowPrivCloneUserns	note: 100% of failures is on centos-7-noselinux-devicemapper
✅ #35336	119	DockerDaemonSuite.TestDaemonUserNamespaceRootSetting	#35336 note: 100% of failures is on centos-7-noselinux-devicemapper
	116	DockerDaemonSuite.TestRemoveContainerAfterLiveRestore	note: 100% of failures is on rhel72 && overlay
✅ #35998	75	TestCreateServiceMultipleTimes	related: #35998
✅ #36502	59	DockerSwarmSuite.TestServiceUpdatePort	#36501 / #30427 / #36743 test was removed
	55	DockerSwarmSuite.TearDownTest	#33041 / #33805
	44	TestServicePlugin	#37132
	41	DockerSwarmSuite.TestAPISwarmLeaderElection	#32673 (related: #36788)
	39	TestCreateServiceSecretFileMode	#37132
	35	DockerSwarmSuite.TestSwarmClusterRotateUnlockKey	#38885 (related: #33041)
	33	TestCreateServiceConfigFileMode	#36907
	26	DockerSwarmSuite.TestAPISwarmServicesCreate	#36903
	21	DockerSwarmSuite.TestSwarmLockUnlockCluster	#34051 / #35689
	20	DockerSwarmSuite.TestAPISwarmLeaderProxy	#32562
	20	DockerSuite.TestRunOOMExitCode	100% of failures on CentOS and RHEL. Related: #36540
✅ #37438	18	DockerDaemonSuite.TestRunContainerWithBridgeNone	#37246
	16	DockerSwarmSuite.TestSwarmNetworkPluginV2	-
	16	DockerSuite.TestEventsOOMDisableTrue	related: #37220
	15	DockerSwarmSuite.TestAPISwarmRaftQuorum	#34988 / #35039
	14	DockerNetworkSuite.TestDockerPluginV2NetworkDriver	100% of failures on RHEL 7.2 + overlay (related: #28305)
	9	TestServiceRemoveKeepsIngressNetwork	#39426 100% of failures on Ubuntu 16.04 + ZFS
	9	DockerSwarmSuite.TestAPISwarmServicesMultipleAgents	#23626
✅ #36551	8	TestServiceWithPredefinedNetwork	#36547
	7	DockerSwarmSuite.TestAPISwarmNodeDrainPause	#23516
	7	DockerSuite.TestExecInteractiveStdinClose	#36877
	6	DockerSwarmSuite.TestAPISwarmManagerRestore	#34993
	6	DockerSuite.TestLogsTail	-
	6	DockerSuite.TestLogsContainerSmallerThanPage	-
	5	TestIsCopyFileRangeSyscallAvailable	100% of failures on rhel-74-selinux-overlay2-stable
	5	DockerSuite.TestLogsContainerBiggerThanPage	-
	4	TestUpdateMemory	-
	4	DockerSwarmSuite.TestSwarmJoinPromoteLocked	#35263
✅	4	DockerSwarmSuite.TestAPISwarmServicesUpdateStartFirst	#35775 / #35825 #35825 (comment)
	4	DockerSuite.TestRestartContainerwithRestartPolicy	#32219 100% on rhel72 && overlay
	4	DockerSuite.TestLogsTimestamps	-
✅	4	DockerSuite.TestLogsContainerMuchBiggerThanPage	#36500 / #36168
✅ #35982	3	TestDaemonRestartKillContainers	#35914
#27466	3	DockerSwarmSuite.TestSwarmContainerAutoStart	#26588 #29663
✅ #25077	3	DockerSwarmSuite.TestAPISwarmRestartCluster	#24590

Other failures:

Count	Failure	Notes
25	failed to get layer	100% of failures on Ubuntu 16.04, either with BTRFS or ZFS
10	access denied	100% of failures on Ubuntu 16.04, either with AUFS, BTRFS or ZFS
6	GIT_SHA1: unbound variable	100% of failures on Ubuntu 16.04, either with AUFS, BTRFS or ZFS

[thaJeztah]

/cc @vdemeester @AntaresS @tiborvass

[kolyshkin]

TestMount should be fixed by #36750 (and I can't update the description)

[kolyshkin]

DockerSwarmSuite.TearDownTest is not a test case per se, but rather a cleanup after each test, so when showing a failure from it, one needs to see which test case is causing it -- the problem is with that test case, not the TearDownTest

[thaJeztah]

@kolyshkin thanks! updated 👍

[olljanat]

TestServiceWithDefaultAddressPoolInit looks to be also flaky. Created item #38514

[vvoland]

Closing in favor of the https://github.com/moby/moby/issues?q=is%3Aissue%20state%3Aopen%20label%3Aflaky label