PILOT-2809: update the lineage to use core zone file instead of greenroom

[colorzzr]

Summary

With the workflow update, the lineage feature for cli (ONLY) will specify the file in the core zone instead of greenroom. This operation is not treated as copy_to_core. There will be another type in the metadata service. Also the upload service should be updated as well

JIRA Issues

PILOT-2809

Type of Change

Please delete options that are not relevant.

• Bug fix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds functionality)
• Breaking change (fix or feature that would cause existing functionality to not work as expected)
• This change requires a documentation update
• Refactor or reformatting

Testing

Are there any new or updated tests to validate the changes?

• Yes
• No

Test Directions

[github-actions]

Coverage Report

File	Stmts	Miss	Cover	Missing
__init__.py	0	0	100%
pilotcli.py	0	29	0%	5–6, 8–12, 15–19, 21–26, 29–37, 40–41
commands
__init__.py	0	0	100%
container_registry.py	0	47	0%	5, 7–10, 13–14, 16, 19–24, 27–33, 36–44, 47–53, 56–65
dataset.py	0	48	44%	19, 36–38, 41–48, 50–60, 78–80, 83–91, 93–103, 123, 127
entry_point.py	0	102	0%	5, 7, 9–10, 12–25, 28–39, 41–42, 45–51, 54–56, 59–63, 66–70, 73–77, 80–83, 86–88, 91–94, 97–114, 117, 119–121, 123–129, 131, 133–135, 137
file.py	0	61	68%	36, 133, 137, 155–156, 198, 273–280, 282, 284, 298–304, 306, 335, 337, 340, 376–382, 385–386, 388–389, 391–403, 405–407, 409, 412–414, 416–418
hpc.py	0	123	0%	5, 7–17, 20–21, 23, 26–29, 36–40, 42–45, 47–49, 51, 53–61, 64–72, 75–84, 87–100, 103–117, 120–132, 134–137, 140–153, 155–157
kg_resource.py	0	13	0%	5, 7–9, 12–13, 15, 18–23
project.py	0	39	0%	5–6, 8–11, 14–15, 17, 20–23, 31, 39, 48–52, 55–62, 64–74
use_config.py	0	13	0%	5, 7–9, 12–13, 15, 18–20, 27–29
user.py	0	30	0%	5, 7–9, 12–15, 18–19, 21, 42–48, 50, 52, 55–56, 58, 61–62, 71–75
configs
__init__.py	0	0	100%
app_config.py	0	0	100%
user_config.py	0	15	79%	49, 59, 63, 67, 71, 75, 79, 83, 87, 91, 95, 99, 103, 107, 111
models
__init__.py	0	0	100%
convert_type.py	0	8	0%	5, 7, 10–15
service_meta_class.py	0	2	80%	9, 16
singleton.py	0	0	100%
upload_form.py	0	26	61%	10, 25, 29, 33, 37, 41, 45, 49, 53, 57, 61, 65, 69, 73, 77, 81, 85, 89, 93, 97, 122, 134–138
resources
custom_error.py	0	0	100%
custom_help.py	0	0	100%
services
__init__.py	0	0	100%
services/crypto
__init__.py	0	0	100%
crypto.py	0	19	40%	35, 43–47, 49, 59–61, 69–75, 77, 79
services/dataset_manager
dataset_detail.py	0	23	70%	42, 44, 61, 68–73, 75–77, 79–88, 90
dataset_download.py	0	74	41%	50, 54, 58–59, 64–70, 73–74, 76–77, 80–85, 87–88, 92–97, 99, 103–107, 111–113, 115–116, 118–121, 129–132, 135–137, 139–141, 143–144, 146–148, 152–159, 161, 167–171, 173
dataset_list.py	0	6	83%	37, 43–44, 46–48
model.py	0	1	87%	15
services/file_manager
__init__.py	0	0	100%
file_list.py	0	19	71%	29, 48, 50, 57–58, 63–65, 73–76, 78, 84–89
file_manifests.py	0	95	25%	19–24, 39–43, 47–48, 51–59, 61, 65–68, 71–75, 77, 81–82, 85–87, 91–92, 95–102, 104, 107, 109–110, 112–113, 115–118, 123–129, 134–139, 142–148, 150–152, 155–161, 163–165, 167–170
file_tag.py	0	39	31%	23, 27–31, 33, 36–48, 50–52, 54–55, 59–61, 64–68, 70–73, 75–76
services/file_manager/file_download
__init__.py	0	0	100%
download_client.py	0	187	17%	29–41, 44–45, 47–53, 56–57, 59–60, 69, 71–72, 76–79, 86, 91–94, 96, 98–106, 108, 110, 114–119, 121, 124–125, 128–130, 132–134, 136–137, 139–141, 145–153, 156–161, 165–174, 182–184, 186–193, 195–199, 204–214, 216–217, 220–221, 226–227, 231–236, 240–241, 244–246, 248–259, 263–266, 269–273, 275–277, 279–281, 283–284, 286, 290–292, 294–307, 309
model.py	0	1	87%	15
services/file_manager/file_upload
__init__.py	0	0	100%
exception.py	0	1	75%	10
file_upload.py	0	30	81%	33–40, 74, 104, 135–136, 140, 145, 176, 184–186, 190, 207, 209, 213, 224–225, 227–229, 307, 310, 314
models.py	0	5	91%	23, 41, 147–149
upload_client.py	0	75	56%	98–101, 163–165, 179–185, 187–191, 193–205, 207, 250, 253, 257, 259–262, 266–271, 273, 277, 279, 281, 283, 301, 326–327, 332, 336–339, 343, 359, 361–363, 370, 375–376, 378, 380–381, 383–386, 388
upload_validator.py	0	24	52%	26–28, 31–36, 39–48, 51–52, 57, 59, 61
services/hpc_manager
hpc_auth.py	0	0	100%
hpc_cluster.py	0	68	32%	34–37, 39–40, 42, 46–58, 60–64, 66–67, 72–74, 76–77, 81–93, 95–99, 101–102, 106–118, 120–121, 123
services/kg_manager
kg_resource.py	0	11	85%	33–34, 36–37, 65–66, 81–82, 87–89
services/logger_services
__init__.py	0	0	100%
log_functions.py	0	0	100%
services/output_manager
__init__.py	0	0	100%
error_handler.py	0	1	98%	104
help_page.py	0	12	86%	36–37, 46–47, 59–60, 106–107, 116–117, 129–130
message_handler.py	0	64	59%	17–22, 27, 32, 37, 47, 52–54, 59, 64, 69, 74, 87, 92, 99, 104, 109, 113, 129, 149, 159, 169, 188, 196, 210–221, 232–237, 239–245, 256, 260–261, 265–266, 268–269, 273, 277, 281
response_handler.py	0	53	0%	5–6, 9–12, 14–15, 17–21, 23–24, 26–34, 36–39, 42–45, 47–48, 50–56, 58–59, 61–63, 65–68, 71–73
services/project_manager
__init__.py	0	0	100%
project.py	0	6	83%	38, 44–45, 47–49
services/user_authentication
__init__.py	0	0	100%
decorator.py	0	13	67%	29, 32, 49–53, 55, 59–62, 64
token_manager.py	0	21	57%	25, 28–30, 33, 36–37, 40–41, 61, 65, 69–70, 76–77, 79–82, 84–85
user_login_logout.py	0	43	65%	40–43, 114, 118–124, 126–128, 132–133, 137–144, 146–148, 152–153, 160–165, 167–169, 173–176
user_set_config.py	0	9	47%	15–18, 22–24, 26–27
utils
__init__.py	0	0	100%
aggregated.py	0	31	66%	22, 46, 71–73, 75–76, 83–84, 86, 90–93, 95–96, 115–117, 119, 124–131, 133–135
TOTAL	3071	1487	51%

[colorzzr]

@andricDu thx Dusan! can you also help to deploy this version into staging vm?