From 482c20570ff6bc2bd508aef30755740e67dc501f Mon Sep 17 00:00:00 2001 From: Tushar-TG-14 Date: Thu, 26 Jun 2025 18:28:12 +0530 Subject: [PATCH 1/4] DOC-2746-Log Files page updated --- modules/troubleshooting/pages/log-files.adoc | 91 +++++++++++++------- 1 file changed, 59 insertions(+), 32 deletions(-) diff --git a/modules/troubleshooting/pages/log-files.adoc b/modules/troubleshooting/pages/log-files.adoc index f79217009..40451b8dc 100644 --- a/modules/troubleshooting/pages/log-files.adoc +++ b/modules/troubleshooting/pages/log-files.adoc @@ -1,24 +1,30 @@ = Log Files -The TigerGraph database captures key information on activities occurring across its different components through log functions that output to log files. -These log files are not only helpful in xref:troubleshooting-guide.adoc[troubleshooting] but also serve as a resource for auditing. +TigerGraph captures key information about activities across its components through log files. These logs are essential for xref:troubleshooting:troubleshooting-guide.adoctroubleshooting and auditing. Logs may contain sensitive information, so direct access is restricted. -This page provides a general overview of the way log files are stored in TigerGraph. - +This page provides an overview of the log files available in TigerGraph, including where to find them, how they are stored, and what information they contain. xref:audit-log.adoc[Audit logs] are structured in JSON, ensuring machine-readability and facilitating easy integration with third-party tools. -The TigerGraph Linux server admin user may also use the xref:gcollect.adoc[gcollect] utility to search for and gather selected information from the logs. +TigerGraph Linux server administrators can also use the xref:gcollect.adoc[gcollect] utility to search for and gather selected information from the logs. We also provide instructions on how to xref:elk-filebeat.adoc[set up log viewing with Elasticsearch, Kibana, or Filebeat]. +== Available Log Files -== TigerGraph log structure +TigerGraph generates a variety of log files for its different components. +Understanding what logs are available and what they contain is the first step in effective troubleshooting and system monitoring. + +=== Log File Locations + +Logs in TigerGraph are stored in the log root directory, which is configured at install time. You can find this location by running: -Logs in TigerGraph are stored in TigerGraph's log root directory, which is configured at install time. -You can find the location by running the console command `gadmin config get System.LogRoot`. +[source,console] +---- +gadmin config get System.LogRoot +---- -Within this directory are separate directories for the various TigerGraph services: +Within this directory, you will find subdirectories for each TigerGraph component (e.g., admin, gpe, gsql, gui, kafka, nginx, zk, etc.). [source,console] ---- @@ -27,39 +33,58 @@ admin dict executor gpe gsql informant kafkaconn nginx zk controller etcd fileLoader gse gui kafka kafkastrm-ll restpp ---- -You can also use the `gadmin log` command to list log files: +To list log files, use: -[source, console] +[source,console] +---- +gadmin log +---- + +To get logs for a specific service, use: + +[source,console] ---- -$ gadmin log -ADMIN : /home/tigergraph/tigergraph/log/admin/ADMIN#1.out -ADMIN : /home/tigergraph/tigergraph/log/admin/ADMIN.INFO -CTRL : /home/tigergraph/tigergraph/log/controller/CTRL#1.log -CTRL : /home/tigergraph/tigergraph/log/controller/CTRL#1.out -... -ZK : /home/tigergraph/tigergraph/log/zk/ZK#1.out -ZK : /home/tigergraph/tigergraph/log/zk/zookeeper.log +gadmin log ---- -Use the command `gadmin log ` to just get the logs for a specific service: +Some components like Zookeeper and Kafka have logs that are not listed by `gadmin log`. You can find them at: -[source, console] +[source,console] ---- -$ gadmin log gpe -GPE : /home/tigergraph/tigergraph/log/gpe/GPE_1#1.out -GPE : /home/tigergraph/tigergraph/log/gpe/log.INFO +zookeeper : ~/tigergraph/zk/zookeeper.out.* +kafka : ~/tigergraph/kafka/kafka.out ---- -The `log.INFO` file contains messages logged by the application code. -The `.out` log contains the redirection of the process output, and is used for debugging significantly less frequently than `log.INFO`. +=== Log File Extensions + +* `.out` files capture *standard output (stdout)* and log runtime information, including error stack traces when services crash or unexpected errors occur. +These logs are especially useful for errors that aren't logged by the service's internal logging mechanism. + +* `.ERROR` files are used to log errors captured by the system, typically from exceptions caught in try-catch blocks. If an error occurs before the logging system initializes or is uncaught, it is logged in the `.out` file instead. + +* `.INFO` files log regular operational information about the system's normal functioning. + +To diagnose an issue for a given component, check the `.out` log file for that component. + +image::https://lh5.googleusercontent.com/6MnNakec5fKh5faCoWdZwfzprqXyguDZXt15nz0QAG1M3vW1t0nmwo7oYr3DgwVsgJoIEjub-5tSA81UtOQ-Ot-9m30zZ9Zr5tRG077dgfZ7KaE3tMMafUK63oi6fILQeM-kQw6fKqc[] + +=== Component-Specific Log Details + +* *NGINX Logs:* The NGINX log files (e.g., `nginx.out`, `nginx.error.log`, `nginx.access.log`) are generated directly by the NGINX web server itself and are not internal TigerGraph component logs. + +* *GUI Logs (VIS):* For the GUI component, the `.out file` (e.g., `gui_ADMIN.log`, `gui_INFO.log`) serves as the primary log file. +It captures the standard output of the GUI process and includes all log levels (error, warning, info). There are no other separate log files for the GUI. + +== TigerGraph log structure [CAUTION] +==== The log format differs between the `.out` and `INFO` logs. It also differs between certain TigerGraph services. An internal project to unify log formats is ongoing. +==== -Log formats also vary across the different components. -In folders where logs are checked often, such as `restpp`, `gsql`, and `admin`, there are symbolic links that help you quickly get to the most recent log file of that category: +In directories with frequent log checks (such as `restpp`, `gsql`, and `admin`), symbolic links help you quickly access the most recent log file. * `log.INFO` ** Contains regular output and errors @@ -75,9 +100,11 @@ Historical logs have the form `-old-YYYY-MM-DDTHH-MM-SS.fff.out` ** Contains outputs for any fatal level events [NOTE] +==== All services do not create a `log.DEBUG` file by default. To change this, modify the parameter `.BasicConfig.LogConfig.LogLevel`. For example, `GSQL.BasicConfig.LogConfig.LogLevel`. See xref:reference:configuration-parameters.adoc[] for more information. +==== == Log locations on a cluster @@ -100,10 +127,10 @@ I@20210709 13:56:52.220 (SessionManager.java:204) All sessions aborted. I@20210709 13:56:52.224 (GsqlHAHandler.java:283) switched to new leader m1 ---- - == Open source TigerGraph components -The open source components that TigerGraph includes (Kafka, Nginx, ZooKeeper, Kafkaconn, Kafkastream) follow their respective logging behavior instead of having an `INFO/WARNING/ERROR` log, in addition to having an `.out` file for process output redirection. +For open source components included with TigerGraph (Kafka, Nginx, ZooKeeper, Kafkaconn, Kafkastream), log files follow their respective logging conventions. + For example, the Kafka logs have a `controller.log`, `kafka.log`, `kafka-request.log`, `state-change.log`, and `server.log`. == Log rotation @@ -112,6 +139,6 @@ TigerGraph also handles log rotation. When the log is rotated, the log.LEVEL symlink is updated to point to the newest log. The default configuration is to rotate under any of the following circumstances: -* Log file max size exceeds 100mb +* Log file size exceeds 100 MB * Log is older than 90 days -* There are more than 100 files for that service +* More than 100 files exist for that service From b323286b162465f23e4ef78be5d5095c217e8a97 Mon Sep 17 00:00:00 2001 From: Tushar-TG-14 Date: Wed, 9 Jul 2025 14:27:40 +0530 Subject: [PATCH 2/4] DOC-2746-Final Updates --- modules/troubleshooting/pages/log-files.adoc | 105 +++++++++++-------- 1 file changed, 64 insertions(+), 41 deletions(-) diff --git a/modules/troubleshooting/pages/log-files.adoc b/modules/troubleshooting/pages/log-files.adoc index 40451b8dc..f54070f7e 100644 --- a/modules/troubleshooting/pages/log-files.adoc +++ b/modules/troubleshooting/pages/log-files.adoc @@ -1,12 +1,12 @@ = Log Files -TigerGraph captures key information about activities across its components through log files. These logs are essential for xref:troubleshooting:troubleshooting-guide.adoctroubleshooting and auditing. +TigerGraph captures key information about activities across its components through log files. These logs are essential for xref:troubleshooting:troubleshooting-guide.adoc[troubleshooting] and auditing. Logs may contain sensitive information, so direct access is restricted. This page provides an overview of the log files available in TigerGraph, including where to find them, how they are stored, and what information they contain. xref:audit-log.adoc[Audit logs] are structured in JSON, ensuring machine-readability and facilitating easy integration with third-party tools. -TigerGraph Linux server administrators can also use the xref:gcollect.adoc[gcollect] utility to search for and gather selected information from the logs. +TigerGraph Linux admin users also use the xref:gcollect.adoc[gcollect] utility to search for and gather selected information from the logs. We also provide instructions on how to xref:elk-filebeat.adoc[set up log viewing with Elasticsearch, Kibana, or Filebeat]. @@ -24,7 +24,7 @@ Logs in TigerGraph are stored in the log root directory, which is configured at gadmin config get System.LogRoot ---- -Within this directory, you will find subdirectories for each TigerGraph component (e.g., admin, gpe, gsql, gui, kafka, nginx, zk, etc.). +Within this directory, you will find subdirectories for each TigerGraph component (admin, gpe, gsql, gui, kafka, nginx, zk, etc.). [source,console] ---- @@ -33,18 +33,27 @@ admin dict executor gpe gsql informant kafkaconn nginx zk controller etcd fileLoader gse gui kafka kafkastrm-ll restpp ---- -To list log files, use: +Use the `gadmin log` command to list log files: -[source,console] +[source, console] ---- -gadmin log +$ gadmin log +ADMIN : /home/tigergraph/tigergraph/log/admin/ADMIN#1.out +ADMIN : /home/tigergraph/tigergraph/log/admin/ADMIN.INFO +CTRL : /home/tigergraph/tigergraph/log/controller/CTRL#1.log +CTRL : /home/tigergraph/tigergraph/log/controller/CTRL#1.out +... +ZK : /home/tigergraph/tigergraph/log/zk/ZK#1.out +ZK : /home/tigergraph/tigergraph/log/zk/zookeeper.log ---- -To get logs for a specific service, use: +Use the command `gadmin log ` to get the logs for a specific service: -[source,console] +[source, console] ---- -gadmin log +$ gadmin log gpe +GPE : /home/tigergraph/tigergraph/log/gpe/GPE_1#1.out +GPE : /home/tigergraph/tigergraph/log/gpe/log.INFO ---- Some components like Zookeeper and Kafka have logs that are not listed by `gadmin log`. You can find them at: @@ -55,7 +64,7 @@ zookeeper : ~/tigergraph/zk/zookeeper.out.* kafka : ~/tigergraph/kafka/kafka.out ---- -=== Log File Extensions +=== TigerGraph Component Log Files * `.out` files capture *standard output (stdout)* and log runtime information, including error stack traces when services crash or unexpected errors occur. These logs are especially useful for errors that aren't logged by the service's internal logging mechanism. @@ -68,42 +77,66 @@ To diagnose an issue for a given component, check the `.out` log file for that c image::https://lh5.googleusercontent.com/6MnNakec5fKh5faCoWdZwfzprqXyguDZXt15nz0QAG1M3vW1t0nmwo7oYr3DgwVsgJoIEjub-5tSA81UtOQ-Ot-9m30zZ9Zr5tRG077dgfZ7KaE3tMMafUK63oi6fILQeM-kQw6fKqc[] -=== Component-Specific Log Details +[NOTE] +==== +* The GUI component does not have separate `.log`, `.error`, or `.info` files. +* Each GUI log file (e.g., `gui_ADMIN.log`, `gui_INFO.log`) captures the standard output of the GUI process and includes all log levels (error, warning, info). +* The log level for the GUI component is *configurable*. You can set it using: + +[source,console] +---- +gadmin config set GUI.BasicConfig.LogConfig.LogLevel +---- + +Replace `` with one of: `DEBUG`, `INFO`, `WARN`, `ERROR`, `PANIC`, or `FATAL`. The default level is `INFO`. +==== + +==== Symbolic Links + +In directories with frequently checked logs, such as `restpp`, `gsql`, and `admin`, symbolic links make it easier to access the latest log file. +These links are automatically updated to point to the newest log. + +For example, `log.INFO` is a symbolic link that points to the current `.INFO` log file. To see what a symbolic link points to, use `ls -ll` followed by the symbolic link name: + +[source,console] +---- +ls -ll log.INFO +log.INFO -> log.INFO.2024-07-01-10-00-00 +---- + +Here, `log.INFO` is a symbolic link pointing to the current `.INFO` log file. + +=== Third-Party Component Log Files + +TigerGraph uses several open-source components (such as Kafka, Nginx, ZooKeeper, Kafkaconn, Kafkastream) that maintain their own log conventions. * *NGINX Logs:* The NGINX log files (e.g., `nginx.out`, `nginx.error.log`, `nginx.access.log`) are generated directly by the NGINX web server itself and are not internal TigerGraph component logs. -* *GUI Logs (VIS):* For the GUI component, the `.out file` (e.g., `gui_ADMIN.log`, `gui_INFO.log`) serves as the primary log file. -It captures the standard output of the GUI process and includes all log levels (error, warning, info). There are no other separate log files for the GUI. +* *Kafka Logs:* Kafka logs include `controller.log`, `kafka.log`, `kafka-request.log`, `state-change.log`, and `server.log`. + +* *ZooKeeper Logs:* ZooKeeper logs are typically found as `zookeeper.out.*` in the ZooKeeper directory. == TigerGraph log structure [CAUTION] ==== -The log format differs between the `.out` and `INFO` logs. -It also differs between certain TigerGraph services. +Log formats may differ between `.out` and `.INFO` logs and between different TigerGraph services. An internal project to unify log formats is ongoing. ==== -In directories with frequent log checks (such as `restpp`, `gsql`, and `admin`), symbolic links help you quickly access the most recent log file. - -* `log.INFO` -** Contains regular output and errors -* `log.ERROR` -** Contains errors only -* `.out` -** Contains all output from the component process. Current `.out` logs have the form `.out`. -Historical logs have the form `-old-YYYY-MM-DDTHH-MM-SS.fff.out` - +* `log.INFO`: Contains regular output and errors. +* `log.ERROR`: Contains errors only. +* `.out`: Contains all output from the component process. Current `.out` logs have the form `.out`. Historical logs have the form `-old-YYYY-MM-DDTHH-MM-SS.fff.out` * `log.WARNING` or `log.DEBUG` -** `log.WARNING` contains warnings and all error level messages -* `log.FATAL` -** Contains outputs for any fatal level events +** `log.WARNING` contains warnings and all error-level messages. +** `log.DEBUG` contains debug-level messages (not created by default). +* `log.FATAL`: Contains outputs for any fatal level events [NOTE] ==== All services do not create a `log.DEBUG` file by default. To change this, modify the parameter `.BasicConfig.LogConfig.LogLevel`. -For example, `GSQL.BasicConfig.LogConfig.LogLevel`. See xref:reference:configuration-parameters.adoc[] for more information. +For example, `GSQL.BasicConfig.LogConfig.LogLevel`. See xref:reference:configuration-parameters.adoc[Configuration Parameters] for more information. ==== == Log locations on a cluster @@ -127,18 +160,8 @@ I@20210709 13:56:52.220 (SessionManager.java:204) All sessions aborted. I@20210709 13:56:52.224 (GsqlHAHandler.java:283) switched to new leader m1 ---- -== Open source TigerGraph components - -For open source components included with TigerGraph (Kafka, Nginx, ZooKeeper, Kafkaconn, Kafkastream), log files follow their respective logging conventions. - -For example, the Kafka logs have a `controller.log`, `kafka.log`, `kafka-request.log`, `state-change.log`, and `server.log`. - == Log rotation TigerGraph also handles log rotation. -When the log is rotated, the log.LEVEL symlink is updated to point to the newest log. -The default configuration is to rotate under any of the following circumstances: - -* Log file size exceeds 100 MB -* Log is older than 90 days -* More than 100 files exist for that service +When a log is rotated, the symlink (e.g., `log.INFO`) is updated to point to the newest log file. +Logs are rotated when the file size exceeds *100 MB*, the log is older than *90 days*, or more than *100 files* exist for that service. From bd71c292260c3a058df1f3a17da52288ed0dbc00 Mon Sep 17 00:00:00 2001 From: Tushar-TG-14 Date: Fri, 11 Jul 2025 18:19:55 +0530 Subject: [PATCH 3/4] DOC-2746-Final changes --- modules/troubleshooting/pages/log-files.adoc | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/modules/troubleshooting/pages/log-files.adoc b/modules/troubleshooting/pages/log-files.adoc index f54070f7e..ed58f2fab 100644 --- a/modules/troubleshooting/pages/log-files.adoc +++ b/modules/troubleshooting/pages/log-files.adoc @@ -56,7 +56,7 @@ GPE : /home/tigergraph/tigergraph/log/gpe/GPE_1#1.out GPE : /home/tigergraph/tigergraph/log/gpe/log.INFO ---- -Some components like Zookeeper and Kafka have logs that are not listed by `gadmin log`. You can find them at: +Third-Party components like Zookeeper and Kafka have logs that are not listed by `gadmin log`. You can find them at: [source,console] ---- @@ -79,7 +79,8 @@ image::https://lh5.googleusercontent.com/6MnNakec5fKh5faCoWdZwfzprqXyguDZXt15nz0 [NOTE] ==== -* The GUI component does not have separate `.log`, `.error`, or `.info` files. +The GUI component writes all log levels to a single log file and does not generate separate `.log`, `.error`, or `.info` files. + * Each GUI log file (e.g., `gui_ADMIN.log`, `gui_INFO.log`) captures the standard output of the GUI process and includes all log levels (error, warning, info). * The log level for the GUI component is *configurable*. You can set it using: @@ -87,7 +88,7 @@ image::https://lh5.googleusercontent.com/6MnNakec5fKh5faCoWdZwfzprqXyguDZXt15nz0 ---- gadmin config set GUI.BasicConfig.LogConfig.LogLevel ---- - ++ Replace `` with one of: `DEBUG`, `INFO`, `WARN`, `ERROR`, `PANIC`, or `FATAL`. The default level is `INFO`. ==== @@ -121,7 +122,6 @@ TigerGraph uses several open-source components (such as Kafka, Nginx, ZooKeeper, [CAUTION] ==== Log formats may differ between `.out` and `.INFO` logs and between different TigerGraph services. -An internal project to unify log formats is ongoing. ==== * `log.INFO`: Contains regular output and errors. From 9555478a7320eb21d4374ca2822f322894a4698b Mon Sep 17 00:00:00 2001 From: Tushar-TG-14 Date: Fri, 11 Jul 2025 18:20:45 +0530 Subject: [PATCH 4/4] DOC-2746-Update log-files.adoc --- modules/troubleshooting/pages/log-files.adoc | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/modules/troubleshooting/pages/log-files.adoc b/modules/troubleshooting/pages/log-files.adoc index ed58f2fab..3817afd8c 100644 --- a/modules/troubleshooting/pages/log-files.adoc +++ b/modules/troubleshooting/pages/log-files.adoc @@ -88,7 +88,7 @@ The GUI component writes all log levels to a single log file and does not genera ---- gadmin config set GUI.BasicConfig.LogConfig.LogLevel ---- -+ + Replace `` with one of: `DEBUG`, `INFO`, `WARN`, `ERROR`, `PANIC`, or `FATAL`. The default level is `INFO`. ====