Merge pull request #1121 from dedemorton/cherry_pick_FAQ

FAQ additions and fixes after review
elastic · Mar 8, 2016 · ed438ab · ed438ab
2 parents 0bf999d + 661a613
commit ed438ab
Show file tree

Hide file tree

Showing 11 changed files with 152 additions and 11 deletions.
diff --git a/filebeat/docs/faq.asciidoc b/filebeat/docs/faq.asciidoc
@@ -4,8 +4,41 @@
 This section contains frequently asked questions about Filebeat. Also check out the
 https://discuss.elastic.co/c/beats/filebeat[Filebeat discussion forum].
 
-=== Can I read log files from network volumes?
+[[filebeat-network-volumes]]
+=== Why can't I read log files from network volumes?
 
 We do not recommend reading log files from network volumes. Whenever possible, install Filebeat on the host machine and
 send the log files directly from there. Reading files from network volumes (especially on Windows) can have unexpected side
 effects. For example, changed file identifiers may result in Filebeat reading a log file from scratch again.
+
+=== Why isn’t Filebeat collecting lines from my file?
+
+Filebeat might be incorrectly configured or unable to send events to the output. To resolve the issue:
+
+* Make sure the config file specifies the correct path to the file that you are collecting. See <<filebeat-configuration>>
+for more information.
+* Verify that the file is not older than the value specified by <<ignore-older,`ignore_older`>>. By default, Filebeat
+stops reading files that are older than 24 hours. You can change this behavior by specifying a different value for
+<<ignore-older,`ignore_older`>>.
+* Make sure that Filebeat is able to send events to the configured output. Run Filebeat in debug mode to determine whether
+it's publishing events successfully:
++
+["source","sh",subs="attributes,callouts"]
+----------------------------------------------------------------------
+./filebeat -c config.yml -e -d "*"
+----------------------------------------------------------------------
+
+[[filebeat-cpu]]
+=== Why is Filebeat using too much CPU?
+
+Filebeat might be configured to scan for files too frequently. Check the setting for `scan_frequency` in the `filebeat.yml`
+config file. Setting `scan_frequency` to less than 1s may cause Filebeat to scan the disk in a tight loop.
+
+//=== Why is Filebeat keeping old files open?
+
+[[dashboard-fields-incorrect-filebeat]]
+=== Why is the dashboard in Kibana breaking up my data fields incorrectly?
+
+The index template might not be loaded correctly. See <<filebeat-template>>.
+
+include::../../libbeat/docs/shared-faq.asciidoc[]
diff --git a/filebeat/docs/index.asciidoc b/filebeat/docs/index.asciidoc
@@ -18,7 +18,7 @@ include::./multiple-prospectors.asciidoc[]
 
 include::./troubleshooting.asciidoc[]
 
-//include::./faq.asciidoc[]
+include::./faq.asciidoc[]
 
 include::./migration.asciidoc[]
 

diff --git a/filebeat/docs/reference/configuration/filebeat-options.asciidoc b/filebeat/docs/reference/configuration/filebeat-options.asciidoc
@@ -115,6 +115,7 @@ If this option is set to true, the custom <<configuration-fields>> are stored as
 in the output document instead of being grouped under a `fields` sub-dictionary.
 If the custom field names conflict with other field names added by Filebeat, the custom fields overwrite the other fields.
 
+[[ignore-older]]
 ===== ignore_older
 
 If this option is specified, Filebeat

diff --git a/libbeat/docs/shared-faq.asciidoc b/libbeat/docs/shared-faq.asciidoc
@@ -0,0 +1,70 @@
+//////////////////////////////////////////////////////////////////////////
+//// This content is shared by all Elastic Beats. Make sure you keep the
+//// descriptions here generic enough to work for all Beats that include
+//// this file. When using cross references, make sure that the cross
+//// references resolve correctly for any files that include this one.
+//// Use the appropriate variables defined in the index.asciidoc file to
+//// resolve Beat names: beatname_uc and beatname_lc.
+//// Use the following include to pull this content into a doc file:
+//// include::../../libbeat/docs/shared-faq.asciidoc[]
+//////////////////////////////////////////////////////////////////////////
+
+[[connection-problem]]
+=== Why doesn't my connection to Logstash work?
+
+You may have configured Logstash or the Beat incorrectly. To resolve the issue: 
+
+* Make sure that Logstash is running and you can connect to it. First, try to ping the Logstash host to verify that you can reach it
+from the host running the Beat. Then use either `nc` or `telnet` to make sure that the port is available. For example:
++
+[source,shell]
+----------------------------------------------------------------------
+ping <hostname or IP>
+telnet <hostname or IP> 5044
+----------------------------------------------------------------------
+* Verify that the config file for your Beat specifies the correct port where Logstash is running.  
+* Make sure that the Elasticsearch output is commented out in the config file and the Logstash output is uncommented. 
+* Confirm that the most recent Beats input plugin for Logstash is installed and configured. Note that Beats will not connect
+to the Lumberjack input plugin. See
+{libbeat}/logstash-installation.html#logstash-input-update[Updating the Beats Input Plugin for Logstash].
+
+[[metadata-missing]]
+=== Why is @metadata missing in Logstash?
+
+Logstash outputs remove `@metadata` fields automatically. Therefore, if Logstash instances are chained directly or via some message
+queue (for example, Redis or Kafka), the `@metadata` field will not be available in the final Logstash instance.
+
+TIP: To preserve `@metadata` fields, use the Logstash mutate filter with the rename setting to rename the fields to
+non-internal fields.
+
+[[diff-logstash-beats]]
+=== What is the difference between Logstash and Beats?
+
+Beats are lightweight data shippers that you install as agents on your servers to send specific types of operational
+data to Elasticsearch. Beats have a small footprint and use fewer system resources than Logstash.
+
+Logstash has a larger footprint, but provides a broad array of input, filter, and output plugins for collecting, enriching,
+and transforming data from a variety of sources. 
+
+For more information, see the https://www.elastic.co/guide/en/logstash/current/introduction.html[Logstash Introduction] and
+the https://www.elastic.co/guide/en/beats/libbeat/current/beats-reference.html[Beats Overview].
+
+[[ssl-client-fails]]
+=== Why does my SSL client fail to connect to Logstash?
+
+The host running Logstash might be unreachable or the certificate may not be valid. To resolve your issue:
+
+* Make sure that Logstash is running and you can connect to it. First, try to ping the Logstash host to verify that you can reach it
+from the host running the Beat. Then use either `nc` or `telnet` to make sure that the port is available. For example:
++
+[source,shell]
+----------------------------------------------------------------------
+ping <hostname or IP>
+telnet <hostname or IP> 5044
+----------------------------------------------------------------------
+
+* Verify that the certificate is valid and that the hostname and IP match. 
++
+TIP: For testing purposes only, you can set `insecure: true` to disable hostname checking.
+
+* Make sure that you have enabled SSL (set `ssl => true`) when configuring the https://www.elastic.co/guide/en/logstash/current/plugins-inputs-beats.html[Beats input plugin for Logstash].
diff --git a/packetbeat/docs/faq.asciidoc b/packetbeat/docs/faq.asciidoc
@@ -4,9 +4,40 @@
 This section contains frequently asked questions about Packetbeat. Also check out the
 https://discuss.elastic.co/c/beats/packetbeat[Packetbeat discussion forum].
 
+[[client-server-fields-empty]]
 === Why are the client_server and server fields empty?
 
-The `client_server` and `server` fiels will be empty if you have not configured Packetbeat
+The `client_server` and `server` fields are empty when Packetbeat is not configured
 to capture information about the network topology.
 
-To capture information about the network topology, you need to set the `save_topology` configuration option to true and make sure that you are sending the output to Elasticsearch.
+To capture information about the network topology, set the `save_topology` configuration option to true and make sure that
+you are sending the output to Elasticsearch.
+
+[[dashboard-fields-incorrect]]
+=== Why is the dashboard in Kibana breaking up my data fields incorrectly?
+
+The index template might not be loaded correctly. See <<packetbeat-template>>.
+
+[[packetbeat-mirror-ports]]
+=== Why doesn’t Packetbeat see any packets when using mirror ports?
+
+The interface needs to be set to promiscuous mode. Run the following command:
+
+["source","sh",subs="attributes,callouts"]
+----------------------------------------------------------------------
+ip link set <device_name> promisc on
+----------------------------------------------------------------------
+
+For example: `ip link set enp5s0f1 promisc on`
+
+[[packetbeat-missing-transactions]]
+=== Why is Packetbeat missing long running transactions?
+
+Packetbeat has an internal timeout that it uses to time out transactions and TCP connections
+when no packets have been seen for a long time. 
+
+To process long running transactions, you can specify a larger value for the <<transaction-timeout-option,`transaction_timeout`>>
+option. However, keep in mind that very large timeout values can increase memory usage if messages are lost or transaction
+response messages are not sent. 
+
+include::../../libbeat/docs/shared-faq.asciidoc[]
diff --git a/packetbeat/docs/index.asciidoc b/packetbeat/docs/index.asciidoc
@@ -28,7 +28,7 @@ include::./kibana3.asciidoc[]
 
 include::./troubleshooting.asciidoc[]
 
-//include::./faq.asciidoc[]
+include::./faq.asciidoc[]
 
 include::./reference.asciidoc[]
 

diff --git a/packetbeat/docs/reference/configuration/packetbeat-options.asciidoc b/packetbeat/docs/reference/configuration/packetbeat-options.asciidoc
@@ -257,6 +257,7 @@ is sent to Elasticsearch. The default is false.  This option is useful when you
 want to index the whole response. Note that for HTTP, the body is not included
 by default, only the HTTP headers.
 
+[[transaction-timeout-option]]
 ===== transaction_timeout
 
 The per protocol transaction timeout. Expired transactions will no longer be correlated to incoming responses, but sent to Elasticsearch immediately.

diff --git a/topbeat/docs/faq.asciidoc b/topbeat/docs/faq.asciidoc
@@ -4,6 +4,9 @@
 This section contains frequently asked questions about Topbeat. Also check out the
 https://discuss.elastic.co/c/beats/topbeat[Topbeat discussion forum].
 
-=== Question 1
+[[dashboard-fields-incorrect-topbeat]]
+=== Why is the dashboard in Kibana breaking up my data fields incorrectly?
 
-Answer TBD.
+The index template might not be loaded correctly. See <<topbeat-template>>.
+
+include::../../libbeat/docs/shared-faq.asciidoc[]
diff --git a/topbeat/docs/index.asciidoc b/topbeat/docs/index.asciidoc
@@ -16,7 +16,7 @@ include::../../libbeat/docs/https.asciidoc[]
 
 include::./troubleshooting.asciidoc[]
 
-//include::./faq.asciidoc[]
+include::./faq.asciidoc[]
 
 include::./reference.asciidoc[]
 

diff --git a/winlogbeat/docs/faq.asciidoc b/winlogbeat/docs/faq.asciidoc
@@ -4,7 +4,9 @@
 This section contains frequently asked questions about Winlogbeat. Also check out the
 https://discuss.elastic.co/c/beats/winlogbeat[Winlogbeat discussion forum].
 
+[[dashboard-fields-incorrect]]
+=== Why is the dashboard in Kibana breaking up my data fields incorrectly?
 
-=== Question 1
+The index template might not be loaded correctly. See <<winlogbeat-template>>.
 
-Answer TBD.
+include::../../libbeat/docs/shared-faq.asciidoc[]
diff --git a/winlogbeat/docs/index.asciidoc b/winlogbeat/docs/index.asciidoc
@@ -16,7 +16,7 @@ include::../../libbeat/docs/https.asciidoc[]
 
 include::./troubleshooting.asciidoc[]
 
-//include::./faq.asciidoc[]
+include::./faq.asciidoc[]
 
 include::./reference.asciidoc[]