1+ ** Note: Maintainers needed : Those that committed in the past code to this repo or are presenting new PRs and have experience and interest on helping to maintain repos & python libraries (code quality, testing, integration, etc). If you are interested on getting our PR's through and helping others to contribute to the library, please get in touch.**
12
2- ===============
3- CMRESHandler.py
4- ===============
3+ ----
54
6- | |license| |versions| |status| |downloads|
7- | |ci_status| |codecov| |gitter|
5+ [ ![ downloads] ( https://img.shields.io/pypi/dd/CMRESHandler.svg )] ( https://pypi.python.org/pypi/CMRESHandler )
6+ [ ![ versions] ( https://img.shields.io/pypi/pyversions/CMRESHandler.svg )] ( https://pypi.python.org/pypi/CMRESHandler )
7+ [ ![ status] ( https://img.shields.io/pypi/status/CMRESHandler.svg )] ( https://pypi.python.org/pypi/CMRESHandler )
8+ [ ![ license] ( https://img.shields.io/pypi/l/CMRESHandler.svg )] ( https://pypi.python.org/pypi/CMRESHandler )
9+ [ ![ ci_status] ( https://travis-ci.org/cmanaha/python-elasticsearch-logger.svg?branch=master )] ( https://travis-ci.org/cmanaha/python-elasticsearch-logger )
10+ [ ![ codecov] ( https://codecov.io/github/cmanaha/python-elasticsearch-logger/coverage.svg?branch=master )] ( http://codecov.io/github/cmanaha/python-elasticsearch-logger?branch=master )
11+ [ ![ gitter] ( https://badges.gitter.im/Join%20Chat.svg )] ( https://gitter.im/cmanaha/python-elasticsearch-logger?utm_source=badge&utm_medium=badge&utm_campaign=pr-badge )
812
9-
10- **Note: Maintainers needed : Those that committed in the past code to this repo or are presenting new PRs and have experience and interest on helping to maintain repos & python libraries (code quality, testing, integration, etc). If you are intereted on getting our PR's through and helping others to contribute to the library, please get in touch. **
11-
12-
13- Python Elasticsearch Log handler
14- ********************************
13+ # Python Elasticsearch Log handler
1514
1615This library provides an Elasticsearch logging appender compatible with the
1716python standard ` logging <https://docs.python.org/2/library/logging.html> ` _ library.
@@ -20,58 +19,62 @@ The code source is in github at `https://github.com/cmanaha/python-elasticsearch
2019< https://github.com/cmanaha/python-elasticsearch-logger > `_
2120
2221
23- Installation
24- ============
22+ # Installation
23+
2524Install using pip::
2625
2726 pip install CMRESHandler
2827
29- Requirements Python 2
30- =====================
28+ # Requirements Python 2
29+
3130This library requires the following dependencies
3231 - elasticsearch
3332 - requests
3433 - enum
3534
3635
37- Requirements Python 3
38- =====================
36+ # Requirements Python 3
37+
3938This library requires the following dependencies
4039 - elasticsearch
4140 - requests
4241
43- Additional requirements for Kerberos support
44- ============================================
45- Additionally, the package support optionally kerberos authentication by adding the following dependecy
42+ # Additional requirements for Kerberos support
43+
44+ Additionally, the package support optionally kerberos authentication by adding the following dependency
4645 - requests-kerberos
4746
48- Additional requirements for AWS IAM user authentication (request signing)
49- =========================================================================
50- Additionally, the package support optionally AWS IAM user authentication by adding the following dependecy
47+ # Additional requirements for AWS IAM user authentication (request signing)
48+
49+ Additionally, the package support optionally AWS IAM user authentication by adding the following dependency
5150 - requests-aws4auth
5251
53- Using the handler in your program
54- ==================================
55- To initialise and create the handler, just add the handler to your logger as follow ::
52+ # Using the handler in your program
5653
57- from cmreslogging.handlers import CMRESHandler
58- handler = CMRESHandler(hosts=[{'host': 'localhost', 'port': 9200}],
59- auth_type=CMRESHandler.AuthType.NO_AUTH,
60- es_index_name="my_python_index")
61- log = logging.getLogger("PythonTest")
62- log.setLevel(logging.INFO)
63- log.addHandler(handler)
54+ To initialize and create the handler, just add the handler to your logger as follow ::
55+
56+ ``` python
57+ from cmreslogging.handlers import CMRESHandler
58+ handler = CMRESHandler(hosts = [{' host' : ' localhost' , ' port' : 9200 }],
59+ auth_type = CMRESHandler.AuthType.NO_AUTH ,
60+ es_index_name = " my_python_index" )
61+ log = logging.getLogger(" PythonTest" )
62+ log.setLevel(logging.INFO )
63+ log.addHandler(handler)
64+ ```
6465
6566You can add fields upon initialisation, providing more data of the execution context ::
6667
67- from cmreslogging.handlers import CMRESHandler
68- handler = CMRESHandler(hosts=[{'host': 'localhost', 'port': 9200}],
69- auth_type=CMRESHandler.AuthType.NO_AUTH,
70- es_index_name="my_python_index",
71- es_additional_fields={'App': 'MyAppName', 'Environment': 'Dev'})
72- log = logging.getLogger("PythonTest")
73- log.setLevel(logging.INFO)
74- log.addHandler(handler)
68+ ``` python
69+ from cmreslogging.handlers import CMRESHandler
70+ handler = CMRESHandler(hosts = [{' host' : ' localhost' , ' port' : 9200 }],
71+ auth_type = CMRESHandler.AuthType.NO_AUTH ,
72+ es_index_name = " my_python_index" ,
73+ es_additional_fields = {' App' : ' MyAppName' , ' Environment' : ' Dev' })
74+ log = logging.getLogger(" PythonTest" )
75+ log.setLevel(logging.INFO )
76+ log.addHandler(handler)
77+ ```
7578
7679This additional fields will be applied to all logging fields and recorded in elasticsearch
7780
@@ -82,18 +85,20 @@ To log, use the regular commands from the logging library ::
8285Your code can also dump additional extra fields on a per log basis that can be used to instrument
8386operations. For example, when reading information from a database you could do something like::
8487
85- start_time = time.time()
86- database_operation()
87- db_delta = time.time() - start_time
88- log.debug("DB operation took %.3f seconds" % db_delta, extra={'db_execution_time': db_delta})
88+ ``` python
89+ start_time = time.time()
90+ database_operation()
91+ db_delta = time.time() - start_time
92+ log.debug(" DB operation took %.3f seconds" % db_delta, extra = {' db_execution_time' : db_delta})
93+ ```
8994
9095The code above executes the DB operation, measures the time it took and logs an entry that contains
9196in the message the time the operation took as string and for convenience, it creates another field
9297called db_execution_time with a float that can be used to plot the time this operations are taking using
9398Kibana on top of elasticsearch
9499
95- Initialisation parameters
96- =========================
100+ # Initialization parameters
101+
97102The constructors takes the following parameters:
98103 - hosts: The list of hosts that elasticsearch clients will connect, multiple hosts are allowed, for example ::
99104
@@ -118,54 +123,56 @@ The constructors takes the following parameters:
118123 - es_doc_type: A string with the name of the document type that will be used `` python_log `` used by default
119124 - es_additional_fields: A dictionary with all the additional fields that you would like to add to the logs
120125
121- Django Integration
122- ==================
126+ # Django Integration
127+
123128It is also very easy to integrate the handler to ` Django <https://www.djangoproject.com/> ` _ And what is even
124129better, at DEBUG level django logs information such as how long it takes for DB connections to return so
125130they can be plotted on Kibana, or the SQL statements that Django executed. ::
126131
127- from cmreslogging.handlers import CMRESHandler
128- LOGGING = {
129- 'version': 1,
130- 'disable_existing_loggers ': False ,
131- 'handlers ': {
132- 'file ': {
133- 'level ': 'DEBUG',
134- 'class ': 'logging.handlers.RotatingFileHandler ',
135- 'filename ': './debug.log ',
136- 'maxBytes ': 102400 ,
137- 'backupCount ': 5 ,
138- } ,
139- 'elasticsearch': {
140- 'level ': 'DEBUG',
141- 'class ': 'cmreslogging.handlers.CMRESHandler ',
142- 'hosts ': [{'host': 'localhost', 'port': 9200}] ,
143- 'es_index_name ': 'my_python_app' ,
144- 'es_additional_fields ': {'App': 'Test', 'Environment': 'Dev'} ,
145- 'auth_type ': CMRESHandler.AuthType.NO_AUTH ,
146- 'use_ssl ': False ,
147- } ,
132+ ``` python
133+ from cmreslogging.handlers import CMRESHandler
134+ LOGGING = {
135+ ' version ' : 1 ,
136+ ' disable_existing_loggers ' : False ,
137+ ' handlers ' : {
138+ ' file ' : {
139+ ' level ' : ' DEBUG ' ,
140+ ' class ' : ' logging.handlers.RotatingFileHandler ' ,
141+ ' filename ' : ' ./debug.log ' ,
142+ ' maxBytes ' : 102400 ,
143+ ' backupCount ' : 5 ,
144+ },
145+ ' elasticsearch ' : {
146+ ' level ' : ' DEBUG ' ,
147+ ' class ' : ' cmreslogging.handlers.CMRESHandler ' ,
148+ ' hosts ' : [{ ' host ' : ' localhost ' , ' port ' : 9200 }] ,
149+ ' es_index_name ' : ' my_python_app ' ,
150+ ' es_additional_fields ' : { ' App ' : ' Test ' , ' Environment ' : ' Dev ' } ,
151+ ' auth_type ' : CMRESHandler.AuthType. NO_AUTH ,
152+ ' use_ssl ' : False ,
148153 },
149- 'loggers': {
150- 'django ': {
151- 'handlers ': ['file','elasticsearch'],
152- 'level ': 'DEBUG' ,
153- 'propagate ': True ,
154- } ,
154+ },
155+ ' loggers ' : {
156+ ' django ' : {
157+ ' handlers ' : [ ' file ' , ' elasticsearch ' ] ,
158+ ' level ' : ' DEBUG ' ,
159+ ' propagate ' : True ,
155160 },
156- }
161+ },
162+ }
163+ ```
157164
158165There is more information about how Django logging works in the
159166` Django documentation <https://docs.djangoproject.com/en/1.9/topics/logging//> ` _
160167
161168
162- Building the sources & Testing
163- ------------------------------
169+ ## Building the sources & Testing
170+
164171To create the package follow the standard python setup.py to compile.
165172To test, just execute the python tests within the test folder
166173
167- Why using an appender rather than logstash or beats
168- ---------------------------------------------------
174+ ## Why using an appender rather than logstash or beats
175+
169176In some cases is quite useful to provide all the information available within the LogRecords as it contains
170177things such as exception information, the method, file, log line where the log was generated.
171178
@@ -177,29 +184,6 @@ using `SysLogHandler <https://docs.python.org/3/library/logging.handlers.html#sy
177184` logstash syslog plugin <https://www.elastic.co/guide/en/logstash/current/plugins-inputs-syslog.html> ` _ .
178185
179186
180- Contributing back
181- -----------------
182- Feel free to use this as is or even better, feel free to fork and send your pull requests over.
183-
184-
185- .. |downloads | image :: https://img.shields.io/pypi/dd/CMRESHandler.svg
186- :target: https://pypi.python.org/pypi/CMRESHandler
187- :alt: Daily PyPI downloads
188- .. |versions | image :: https://img.shields.io/pypi/pyversions/CMRESHandler.svg
189- :target: https://pypi.python.org/pypi/CMRESHandler
190- :alt: Python versions supported
191- .. |status | image :: https://img.shields.io/pypi/status/CMRESHandler.svg
192- :target: https://pypi.python.org/pypi/CMRESHandler
193- :alt: Package stability
194- .. |license | image :: https://img.shields.io/pypi/l/CMRESHandler.svg
195- :target: https://pypi.python.org/pypi/CMRESHandler
196- :alt: License
197- .. |ci_status | image :: https://travis-ci.org/cmanaha/python-elasticsearch-logger.svg?branch=master
198- :target: https://travis-ci.org/cmanaha/python-elasticsearch-logger
199- :alt: Continuous Integration Status
200- .. |codecov | image :: https://codecov.io/github/cmanaha/python-elasticsearch-logger/coverage.svg?branch=master
201- :target: http://codecov.io/github/cmanaha/python-elasticsearch-logger?branch=master
202- :alt: Coverage!
203- .. |gitter | image :: https://badges.gitter.im/Join%20Chat.svg
204- :target: https://gitter.im/cmanaha/python-elasticsearch-logger?utm_source=badge&utm_medium=badge&utm_campaign=pr-badge
205- :alt: gitter
187+ ## Contributing back
188+
189+ Feel free to use this as is or even better, feel free to fork and send your pull requests over.
0 commit comments