Merge pull request #299 from pythonspeed/various-documentation-improvements

Various documentation improvements

Author: itamarst

.changelog/185.doc

@@ -0,0 +1 @@
+Improved explanations in report of what it is that Fil tracks, and what a flamegraph tells you.

.changelog/291.doc

@@ -0,0 +1 @@
+Fix bad command name in the API documentation, thanks to @kdebrab.

.changelog/292.misc

@@ -0,0 +1 @@
+Reports now have a "open in new tab" button. Thanks to @petergaultney for the suggestion.

.changelog/298.feature

@@ -59,10 +59,10 @@ $ python yourscript.py --config=myconfig
 Now you would do:
 
 ```console
-$ filprofiler python yourscript.py --config=myconfig
+$ fil-profile python yourscript.py --config=myconfig
 ```
 
-Notice that you're doing `filprofiler `**`python`**, rather than `filprofiler run` as you would if you were profiling the full script.
+Notice that you're doing `fil-profile `**`python`**, rather than `fil-profile run` as you would if you were profiling the full script.
 Only functions running for the duration of the `filprofiler.api.profile()` call will have memory profiling enabled, including of course the function you pass in.
 The rest of the code will run at (close) to normal speed and configuration.
 

docs/src/fil/api.md

@@ -40,7 +40,7 @@ def render_report(output_path: str, now: datetime) -> str:
         margin: 4rem auto;
         font-size: 18px;
     }}
-    blockquote {{ border: 1px; border-color: black; }}
+    blockquote {{ border-width: 1px; border-color: black; border-style: solid; padding: 1em; }}
     div {{
         text-align: center;
     }}
@@ -74,42 +74,66 @@ def render_report(output_path: str, now: datetime) -> str:
 <p><code>{argv}</code><p>
 
 <h2>Profiling result</h2>
-<div><iframe id="peak" src="peak-memory.svg" width="100%" height="400" scrolling="auto" frameborder="0"></iframe><br>
-<p><input type="button" onclick="fullScreen('#peak');" value="Full screen"></p></div>
-
+<div><p><input type="button" onclick="fullScreen('#peak');" value="Full screen"> · <a href="peak-memory.svg" target="_blank"><button>Open in new window</button></a></p>
+<iframe id="peak" src="peak-memory.svg" width="100%" height="400" scrolling="auto" frameborder="0"></iframe><br>
+</div>
+<br>
+<br>
+<hr>
+<br>
+<div><p><input type="button" onclick="fullScreen('#peak-reversed');" value="Full screen"> · <a href="peak-memory-reversed.svg" target="_blank"><button>Open in new window</button></a></p>
+            <iframe id="peak-reversed" src="peak-memory-reversed.svg" width="100%" height="400" scrolling="auto" frameborder="0"></iframe><br>
+</div>
 <br>
 <blockquote><strong>Need help, or does something look wrong?</strong>
 <a href="https://pythonspeed.com/fil/docs/">Read the documentation</a>,
 and if that doesn't help please
 <a href="https://github.com/pythonspeed/filprofiler/issues/new?body={bugreport}">file an issue</a>
 and I'll try to help.</blockquote>
-<br>
-
-<div><iframe id="peak-reversed" src="peak-memory-reversed.svg" width="100%" height="400" scrolling="auto" frameborder="0"></iframe><br>
-<p><input type="button" onclick="fullScreen('#peak-reversed');" value="Full screen"></p></div>
-
 <br>
 <blockquote><strong>Want memory and performance profiling for your production batch jobs?</strong>
-I'm working on a
-<a href="https://pythonspeed.com/products/fil4prod/"
->always-on profiler called Fil4prod</a> that is fast and robust enough to run in production;
-<a href="mailto:[email protected]">send me an email</a> to participate
-in the alpha program.</blockquote>
+I've also created an
+<a href="https://pythonspeed.com/sciagraph/"
+>always-on profiler called Sciagraph</a> that is fast and robust enough to run in production.</blockquote>
 <br>
 
+<h2>Learn how to reduce memory usage</h2>
+
+<p>Need help reducing your data processing application's memory use? Check out tips and tricks <a href="https://pythonspeed.com/memory/">here</a>.</p>
+
 <h2>Understanding the graphs</h2>
 <p>The flame graphs shows the callstacks responsible for allocations at peak.</p>
 
 <p>The wider (and the redder) the bar, the more memory was allocated by that function or its callers.
 If the bar is 100% of width, that's all the allocated memory.</p>
 
+<p>The left-right axis has no meaning!
+The order of frames is somewhat arbitrary, for example beause multiple calls to the same function may well have been merged into a single callstack.
+So you can't tell from the graph which allocations happened first.
+All you're getting is that at peak allocation these time, these stacktraces were responsible for these allocations.
+</p>
+
 <p>The first graph shows the normal callgraph: if <tt>main()</tt> calls <tt>g()</tt> calls <tt>f()</tt>, let's say, then <tt>main()</tt> will be at the top.
 The second graph shows the reverse callgraph, from <tt>f()</tt> upwards.</p>
 
 <p>Why is the second graph useful? If <tt>f()</tt> is called from multiple places, in the first graph it will show up multiple times, at the bottom.
 In the second reversed graph all calls to <tt>f()</tt> will be merged together.</p>
 
-<p>Need help reducing your data processing application's memory use? Check out tips and tricks <a href="https://pythonspeed.com/memory/">here</a>.</p>
+<h2>Understanding what Fil tracks</h2>
+
+<p>Fil measures how much memory has been allocated; this is not the same as how much memory the process is actively using, nor is it the same as memory resident in RAM.</p>
+
+<ul>
+<li>If the data gets dumped from RAM to swap, Fil still counts it but it's not counted as resident in RAM.</li>
+<li>If the memory is a large chunk of all zeros, on Linux no RAM is used by OS until you actually modify that memory, but Fil will still count it.</li>
+<li>If you have memory that only gets freed on garbage collection
+(this will happen if you have circular references in your data structures),
+memory can be freed at inconsistent times across different runs, especially
+if you're using threads.</li>
+</ul>
+
+<p>See <a href="https://pythonspeed.com/articles/measuring-memory-python/">this article</a> for more details.</p>
+
 </body>
 </html>
 """.format(