1-
2- ===============
3- CMRESHandler.py
4- ===============
5-
6- | |license| |versions| |status| |downloads|
7- | |ci_status| |codecov| |gitter|
8-
9-
1+ @@ -0,0 +1,190 @@
2+ @@ -0,0 +1,189 @@
103** Note: Maintainers needed : Those that committed in the past code to this repo or are presenting new PRs and have experience and interest on helping to maintain repos & python libraries (code quality, testing, integration, etc). If you are intereted on getting our PR's through and helping others to contribute to the library, please get in touch.**
114
5+ ----
126
13- Python Elasticsearch Log handler
14- ********************************
7+ [ ![ downloads] ( https://img.shields.io/pypi/dd/CMRESHandler.svg )] ( https://pypi.python.org/pypi/CMRESHandler )
8+ [ ![ versions] ( https://img.shields.io/pypi/pyversions/CMRESHandler.svg )] ( https://pypi.python.org/pypi/CMRESHandler )
9+ [ ![ status] ( https://img.shields.io/pypi/status/CMRESHandler.svg )] ( https://pypi.python.org/pypi/CMRESHandler )
10+ [ ![ license] ( https://img.shields.io/pypi/l/CMRESHandler.svg )] ( https://pypi.python.org/pypi/CMRESHandler )
11+ [ ![ ci_status] ( https://travis-ci.org/cmanaha/python-elasticsearch-logger.svg?branch=master )] ( https://travis-ci.org/cmanaha/python-elasticsearch-logger )
12+ [ ![ codecov] ( https://codecov.io/github/cmanaha/python-elasticsearch-logger/coverage.svg?branch=master )] ( http://codecov.io/github/cmanaha/python-elasticsearch-logger?branch=master )
13+ [ ![ gitter] ( https://badges.gitter.im/Join%20Chat.svg )] ( https://gitter.im/cmanaha/python-elasticsearch-logger?utm_source=badge&utm_medium=badge&utm_campaign=pr-badge )
14+
15+ # Python Elasticsearch Log handler
1516
1617This library provides an Elasticsearch logging appender compatible with the
1718python standard ` logging <https://docs.python.org/2/library/logging.html> ` _ library.
@@ -20,58 +21,62 @@ The code source is in github at `https://github.com/cmanaha/python-elasticsearch
2021< https://github.com/cmanaha/python-elasticsearch-logger > `_
2122
2223
23- Installation
24- ============
24+ # Installation
25+
2526Install using pip::
2627
2728 pip install CMRESHandler
2829
29- Requirements Python 2
30- =====================
30+ # Requirements Python 2
31+
3132This library requires the following dependencies
3233 - elasticsearch
3334 - requests
3435 - enum
3536
3637
37- Requirements Python 3
38- =====================
38+ # Requirements Python 3
39+
3940This library requires the following dependencies
4041 - elasticsearch
4142 - requests
4243
43- Additional requirements for Kerberos support
44- ============================================
45- Additionally, the package support optionally kerberos authentication by adding the following dependecy
44+ # Additional requirements for Kerberos support
45+
46+ Additionally, the package support optionally kerberos authentication by adding the following dependency
4647 - requests-kerberos
4748
48- Additional requirements for AWS IAM user authentication (request signing)
49- =========================================================================
50- Additionally, the package support optionally AWS IAM user authentication by adding the following dependecy
49+ # Additional requirements for AWS IAM user authentication (request signing)
50+
51+ Additionally, the package support optionally AWS IAM user authentication by adding the following dependency
5152 - requests-aws4auth
5253
53- Using the handler in your program
54- ==================================
55- To initialise and create the handler, just add the handler to your logger as follow ::
54+ # Using the handler in your program
55+
56+ To initialize and create the handler, just add the handler to your logger as follow ::
5657
57- from cmreslogging.handlers import CMRESHandler
58- handler = CMRESHandler(hosts=[{'host': 'localhost', 'port': 9200}],
59- auth_type=CMRESHandler.AuthType.NO_AUTH,
60- es_index_name="my_python_index")
61- log = logging.getLogger("PythonTest")
62- log.setLevel(logging.INFO)
63- log.addHandler(handler)
58+ ``` python
59+ from cmreslogging.handlers import CMRESHandler
60+ handler = CMRESHandler(hosts = [{' host' : ' localhost' , ' port' : 9200 }],
61+ auth_type = CMRESHandler.AuthType.NO_AUTH ,
62+ es_index_name = " my_python_index" )
63+ log = logging.getLogger(" PythonTest" )
64+ log.setLevel(logging.INFO )
65+ log.addHandler(handler)
66+ ```
6467
6568You can add fields upon initialisation, providing more data of the execution context ::
6669
67- from cmreslogging.handlers import CMRESHandler
68- handler = CMRESHandler(hosts=[{'host': 'localhost', 'port': 9200}],
69- auth_type=CMRESHandler.AuthType.NO_AUTH,
70- es_index_name="my_python_index",
71- es_additional_fields={'App': 'MyAppName', 'Environment': 'Dev'})
72- log = logging.getLogger("PythonTest")
73- log.setLevel(logging.INFO)
74- log.addHandler(handler)
70+ ``` python
71+ from cmreslogging.handlers import CMRESHandler
72+ handler = CMRESHandler(hosts = [{' host' : ' localhost' , ' port' : 9200 }],
73+ auth_type = CMRESHandler.AuthType.NO_AUTH ,
74+ es_index_name = " my_python_index" ,
75+ es_additional_fields = {' App' : ' MyAppName' , ' Environment' : ' Dev' })
76+ log = logging.getLogger(" PythonTest" )
77+ log.setLevel(logging.INFO )
78+ log.addHandler(handler)
79+ ```
7580
7681This additional fields will be applied to all logging fields and recorded in elasticsearch
7782
@@ -82,18 +87,20 @@ To log, use the regular commands from the logging library ::
8287Your code can also dump additional extra fields on a per log basis that can be used to instrument
8388operations. For example, when reading information from a database you could do something like::
8489
85- start_time = time.time()
86- database_operation()
87- db_delta = time.time() - start_time
88- log.debug("DB operation took %.3f seconds" % db_delta, extra={'db_execution_time': db_delta})
90+ ``` python
91+ start_time = time.time()
92+ database_operation()
93+ db_delta = time.time() - start_time
94+ log.debug(" DB operation took %.3f seconds" % db_delta, extra = {' db_execution_time' : db_delta})
95+ ```
8996
9097The code above executes the DB operation, measures the time it took and logs an entry that contains
9198in the message the time the operation took as string and for convenience, it creates another field
9299called db_execution_time with a float that can be used to plot the time this operations are taking using
93100Kibana on top of elasticsearch
94101
95- Initialisation parameters
96- =========================
102+ # Initialization parameters
103+
97104The constructors takes the following parameters:
98105 - hosts: The list of hosts that elasticsearch clients will connect, multiple hosts are allowed, for example ::
99106
@@ -118,54 +125,56 @@ The constructors takes the following parameters:
118125 - es_doc_type: A string with the name of the document type that will be used `` python_log `` used by default
119126 - es_additional_fields: A dictionary with all the additional fields that you would like to add to the logs
120127
121- Django Integration
122- ==================
128+ # Django Integration
129+
123130It is also very easy to integrate the handler to ` Django <https://www.djangoproject.com/> ` _ And what is even
124131better, at DEBUG level django logs information such as how long it takes for DB connections to return so
125132they can be plotted on Kibana, or the SQL statements that Django executed. ::
126133
127- from cmreslogging.handlers import CMRESHandler
128- LOGGING = {
129- 'version': 1,
130- 'disable_existing_loggers': False,
131- 'handlers': {
132- 'file': {
133- 'level': 'DEBUG',
134- 'class': 'logging.handlers.RotatingFileHandler',
135- 'filename': './debug.log',
136- 'maxBytes': 102400,
137- 'backupCount': 5,
138- },
139- 'elasticsearch': {
140- 'level': 'DEBUG',
141- 'class': 'cmreslogging.handlers.CMRESHandler',
142- 'hosts': [{'host': 'localhost', 'port': 9200}],
143- 'es_index_name': 'my_python_app',
144- 'es_additional_fields': {'App': 'Test', 'Environment': 'Dev'},
145- 'auth_type': CMRESHandler.AuthType.NO_AUTH,
146- 'use_ssl': False,
147- },
134+ ``` python
135+ from cmreslogging.handlers import CMRESHandler
136+ LOGGING = {
137+ ' version' : 1 ,
138+ ' disable_existing_loggers' : False ,
139+ ' handlers' : {
140+ ' file' : {
141+ ' level' : ' DEBUG' ,
142+ ' class' : ' logging.handlers.RotatingFileHandler' ,
143+ ' filename' : ' ./debug.log' ,
144+ ' maxBytes' : 102400 ,
145+ ' backupCount' : 5 ,
148146 },
149- 'loggers': {
150- 'django': {
151- 'handlers': ['file','elasticsearch'],
152- 'level': 'DEBUG',
153- 'propagate': True,
154- },
147+ ' elasticsearch' : {
148+ ' level' : ' DEBUG' ,
149+ ' class' : ' cmreslogging.handlers.CMRESHandler' ,
150+ ' hosts' : [{' host' : ' localhost' , ' port' : 9200 }],
151+ ' es_index_name' : ' my_python_app' ,
152+ ' es_additional_fields' : {' App' : ' Test' , ' Environment' : ' Dev' },
153+ ' auth_type' : CMRESHandler.AuthType.NO_AUTH ,
154+ ' use_ssl' : False ,
155155 },
156- }
156+ },
157+ ' loggers' : {
158+ ' django' : {
159+ ' handlers' : [' file' ,' elasticsearch' ],
160+ ' level' : ' DEBUG' ,
161+ ' propagate' : True ,
162+ },
163+ },
164+ }
165+ ```
157166
158167There is more information about how Django logging works in the
159168` Django documentation <https://docs.djangoproject.com/en/1.9/topics/logging//> ` _
160169
161170
162- Building the sources & Testing
163- ------------------------------
171+ ## Building the sources & Testing
172+
164173To create the package follow the standard python setup.py to compile.
165174To test, just execute the python tests within the test folder
166175
167- Why using an appender rather than logstash or beats
168- ---------------------------------------------------
176+ ## Why using an appender rather than logstash or beats
177+
169178In some cases is quite useful to provide all the information available within the LogRecords as it contains
170179things such as exception information, the method, file, log line where the log was generated.
171180
@@ -177,29 +186,6 @@ using `SysLogHandler <https://docs.python.org/3/library/logging.handlers.html#sy
177186` logstash syslog plugin <https://www.elastic.co/guide/en/logstash/current/plugins-inputs-syslog.html> ` _ .
178187
179188
180- Contributing back
181- -----------------
182- Feel free to use this as is or even better, feel free to fork and send your pull requests over.
183-
184-
185- .. |downloads | image :: https://img.shields.io/pypi/dd/CMRESHandler.svg
186- :target: https://pypi.python.org/pypi/CMRESHandler
187- :alt: Daily PyPI downloads
188- .. |versions | image :: https://img.shields.io/pypi/pyversions/CMRESHandler.svg
189- :target: https://pypi.python.org/pypi/CMRESHandler
190- :alt: Python versions supported
191- .. |status | image :: https://img.shields.io/pypi/status/CMRESHandler.svg
192- :target: https://pypi.python.org/pypi/CMRESHandler
193- :alt: Package stability
194- .. |license | image :: https://img.shields.io/pypi/l/CMRESHandler.svg
195- :target: https://pypi.python.org/pypi/CMRESHandler
196- :alt: License
197- .. |ci_status | image :: https://travis-ci.org/cmanaha/python-elasticsearch-logger.svg?branch=master
198- :target: https://travis-ci.org/cmanaha/python-elasticsearch-logger
199- :alt: Continuous Integration Status
200- .. |codecov | image :: https://codecov.io/github/cmanaha/python-elasticsearch-logger/coverage.svg?branch=master
201- :target: http://codecov.io/github/cmanaha/python-elasticsearch-logger?branch=master
202- :alt: Coverage!
203- .. |gitter | image :: https://badges.gitter.im/Join%20Chat.svg
204- :target: https://gitter.im/cmanaha/python-elasticsearch-logger?utm_source=badge&utm_medium=badge&utm_campaign=pr-badge
205- :alt: gitter
189+ ## Contributing back
190+
191+ Feel free to use this as is or even better, feel free to fork and send your pull requests over.
0 commit comments