241 lines
8.4 KiB
HTML
241 lines
8.4 KiB
HTML
<!DOCTYPE html>
|
|
<html lang="en">
|
|
<head>
|
|
<meta charset="UTF-8">
|
|
<meta name="viewport" content="width=device-width, initial-scale=1.0">
|
|
<meta http-equiv="Content-Security-Policy" content="default-src 'self'; script-src 'self'; style-src 'self'; font-src 'self' data:; img-src 'self' data:; connect-src 'self'; base-uri 'self'; form-action 'self' https://defcon.social https://bsky.app;">
|
|
<meta http-equiv="X-Content-Type-Options" content="nosniff">
|
|
<link rel="stylesheet" href="../assets/css/style.css">
|
|
<link rel="icon" type="image/x-icon" href="../favicon.ico">
|
|
<script>
|
|
// Apply theme immediately to prevent flash
|
|
(function() {
|
|
const theme = localStorage.getItem('theme') ||
|
|
(window.matchMedia && window.matchMedia('(prefers-color-scheme: dark)').matches ? 'dark' : 'light');
|
|
document.documentElement.setAttribute('data-theme', theme);
|
|
})();
|
|
</script>
|
|
<title>miller Cheatsheet - Cheatsheets - Launch Pad</title>
|
|
</head>
|
|
<body>
|
|
<button class="theme-toggle" id="themeToggle" aria-label="Toggle dark mode">
|
|
<svg class="theme-icon theme-icon-moon" xmlns="http://www.w3.org/2000/svg" width="24" height="24" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><path d="M21 12.79A9 9 0 1 1 11.21 3 7 7 0 0 0 21 12.79z"></path></svg>
|
|
<svg class="theme-icon theme-icon-sun" xmlns="http://www.w3.org/2000/svg" width="24" height="24" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" style="display: none;"><circle cx="12" cy="12" r="5"></circle><line x1="12" y1="1" x2="12" y2="3"></line><line x1="12" y1="21" x2="12" y2="23"></line><line x1="4.22" y1="4.22" x2="5.64" y2="5.64"></line><line x1="18.36" y1="18.36" x2="19.78" y2="19.78"></line><line x1="1" y1="12" x2="3" y2="12"></line><line x1="21" y1="12" x2="23" y2="12"></line><line x1="4.22" y1="19.78" x2="5.64" y2="18.36"></line><line x1="18.36" y1="5.64" x2="19.78" y2="4.22"></line></svg>
|
|
</button>
|
|
<br/><br/>
|
|
<div class="name">
|
|
__ _______________________ _________._________________________
|
|
\_ _____/ \______ \ / _ \ / _____/ / _____/ | | \_ _____/
|
|
| __) | _/ / /_\ \ / \ ___ / \ ___ | | | __)_
|
|
| \ | | \ / | \ \ \_\ \ \ \_\ \ | |___ | \
|
|
\___ / |____|_ / \____|__ / \______ / \______ / |_______ \ /_______ /
|
|
\/ \/ \/ \/ \/ \/ \/
|
|
</div>
|
|
<div class="blog-page-header">
|
|
<div class="blog-header-content">
|
|
<a href="/cheatsheets" class="back-link" title="Back to Cheatsheets">
|
|
<svg xmlns="http://www.w3.org/2000/svg" width="42" height="42" viewBox="0 0 24 24" class="home-icon"><path fill="currentColor" d="M10 20v-6h4v6h5v-8h3L12 3 2 12h3v8z"/></svg>
|
|
</a>
|
|
<h1 class="blog-page-title">miller Cheatsheet</h1>
|
|
</div>
|
|
</div>
|
|
<div class="blog-post-container">
|
|
<div class="blog-posts-container" style="max-width: 900px; margin: 0 auto;">
|
|
<div class="blog-post">
|
|
<div class="blog-post-content">
|
|
<p><a href="index.html">← Back to cheatsheets</a></p>
|
|
<p><a href="../index.html">← Home</a></p>
|
|
<hr>
|
|
<p>miller (mlr) is like awk, sed, cut, join, and sort for name-indexed data such as CSV, TSV, and tabular JSON. Stream processing for structured data with powerful transformation capabilities.</p>
|
|
<hr>
|
|
<h2>Basic Usage</h2>
|
|
<ul>
|
|
<li>mlr --csv cat file.csv - Read CSV</li>
|
|
</ul>
|
|
<ul>
|
|
<li>mlr --tsv cat file.tsv - Read TSV</li>
|
|
</ul>
|
|
<ul>
|
|
<li>mlr --json cat file.json - Read JSON</li>
|
|
</ul>
|
|
<ul>
|
|
<li>mlr --csv head -n 10 file.csv - First 10 rows</li>
|
|
</ul>
|
|
<ul>
|
|
<li>mlr --csv tail -n 10 file.csv - Last 10 rows</li>
|
|
</ul>
|
|
<hr>
|
|
<h2>Field Selection</h2>
|
|
<ul>
|
|
<li>mlr --csv cut -f field1,field2 file.csv - Select fields</li>
|
|
</ul>
|
|
<ul>
|
|
<li>mlr --csv cut -x -f field1 file.csv - Exclude field</li>
|
|
</ul>
|
|
<ul>
|
|
<li>mlr --csv cut -o -f field1 file.csv - Only field</li>
|
|
</ul>
|
|
<ul>
|
|
<li>mlr --csv rename field1,newfield1 file.csv - Rename field</li>
|
|
</ul>
|
|
<ul>
|
|
<li>mlr --csv rename -r 'old(.*)', 'new$1' file.csv - Rename with regex</li>
|
|
</ul>
|
|
<hr>
|
|
<h2>Filtering</h2>
|
|
<ul>
|
|
<li>mlr --csv filter '$field == "value"' file.csv - Equal filter</li>
|
|
</ul>
|
|
<ul>
|
|
<li>mlr --csv filter '$field > 10' file.csv - Numeric filter</li>
|
|
</ul>
|
|
<ul>
|
|
<li>mlr --csv filter '$field =~ "pattern"' file.csv - Regex filter</li>
|
|
</ul>
|
|
<ul>
|
|
<li>mlr --csv filter '$field != "value"' file.csv - Not equal</li>
|
|
</ul>
|
|
<ul>
|
|
<li>mlr --csv filter '$field1 == "a" && $field2 > 5' file.csv - Multiple conditions</li>
|
|
</ul>
|
|
<ul>
|
|
<li>mlr --csv filter -x '$field == "value"' file.csv - Exclude matching</li>
|
|
</ul>
|
|
<hr>
|
|
<h2>Transformation</h2>
|
|
<ul>
|
|
<li>mlr --csv put '$new = $old * 2' file.csv - Add computed field</li>
|
|
</ul>
|
|
<ul>
|
|
<li>mlr --csv put '$total = $price * $quantity' file.csv - Calculate</li>
|
|
</ul>
|
|
<ul>
|
|
<li>mlr --csv put '$date = strftime($timestamp, "%Y-%m-%d")' file.csv - Format date</li>
|
|
</ul>
|
|
<ul>
|
|
<li>mlr --csv put '$upper = toupper($field)' file.csv - String functions</li>
|
|
</ul>
|
|
<ul>
|
|
<li>mlr --csv put '$len = length($field)' file.csv - String length</li>
|
|
</ul>
|
|
<hr>
|
|
<h2>Sorting</h2>
|
|
<ul>
|
|
<li>mlr --csv sort -f field1 file.csv - Sort by field</li>
|
|
</ul>
|
|
<ul>
|
|
<li>mlr --csv sort -f field1,field2 file.csv - Sort by multiple fields</li>
|
|
</ul>
|
|
<ul>
|
|
<li>mlr --csv sort -nr -f field1 file.csv - Reverse numeric sort</li>
|
|
</ul>
|
|
<ul>
|
|
<li>mlr --csv sort -f field1 -t field2 file.csv - Sort then tie-break</li>
|
|
</ul>
|
|
<hr>
|
|
<h2>Grouping & Aggregation</h2>
|
|
<ul>
|
|
<li>mlr --csv stats1 -a count -f field1 -g category file.csv - Count by group</li>
|
|
</ul>
|
|
<ul>
|
|
<li>mlr --csv stats1 -a sum -f amount -g category file.csv - Sum by group</li>
|
|
</ul>
|
|
<ul>
|
|
<li>mlr --csv stats1 -a mean -f value -g category file.csv - Mean by group</li>
|
|
</ul>
|
|
<ul>
|
|
<li>mlr --csv stats1 -a min,max -f value -g category file.csv - Min/max</li>
|
|
</ul>
|
|
<ul>
|
|
<li>mlr --csv group-by category file.csv - Group records</li>
|
|
</ul>
|
|
<hr>
|
|
<h2>Joining</h2>
|
|
<ul>
|
|
<li>mlr --csv join -j field1 -f file1.csv file2.csv - Inner join</li>
|
|
</ul>
|
|
<ul>
|
|
<li>mlr --csv join -j field1 -l file1.csv file2.csv - Left join</li>
|
|
</ul>
|
|
<ul>
|
|
<li>mlr --csv join -j field1 -r file1.csv file2.csv - Right join</li>
|
|
</ul>
|
|
<ul>
|
|
<li>mlr --csv join -j field1 -u file1.csv file2.csv - Full outer join</li>
|
|
</ul>
|
|
<hr>
|
|
<h2>Format Conversion</h2>
|
|
<ul>
|
|
<li>mlr --csv --json cat file.csv - CSV to JSON</li>
|
|
</ul>
|
|
<ul>
|
|
<li>mlr --json --csv cat file.json - JSON to CSV</li>
|
|
</ul>
|
|
<ul>
|
|
<li>mlr --csv --tsv cat file.csv - CSV to TSV</li>
|
|
</ul>
|
|
<ul>
|
|
<li>mlr --csv --markdown cat file.csv - CSV to Markdown</li>
|
|
</ul>
|
|
<hr>
|
|
<h2>Common Examples</h2>
|
|
<h3>Select Fields</h3>
|
|
<pre><code>mlr --csv cut -f name,email users.csv</code></pre>
|
|
<p>Extract specific columns.</p>
|
|
<h3>Filter Rows</h3>
|
|
<pre><code>mlr --csv filter '$status == "active"' users.csv</code></pre>
|
|
<p>Filter by condition.</p>
|
|
<h3>Add Field</h3>
|
|
<pre><code>mlr --csv put '$total = $price * $qty' orders.csv</code></pre>
|
|
<p>Calculate new field.</p>
|
|
<h3>Sort</h3>
|
|
<pre><code>mlr --csv sort -f date -nr sales.csv</code></pre>
|
|
<p>Sort by date descending.</p>
|
|
<h3>Group Statistics</h3>
|
|
<pre><code>mlr --csv stats1 -a sum -f amount -g category sales.csv</code></pre>
|
|
<p>Sum by category.</p>
|
|
<h3>Join Files</h3>
|
|
<pre><code>mlr --csv join -j user_id -f users.csv orders.csv</code></pre>
|
|
<p>Join on common field.</p>
|
|
<h3>Convert Format</h3>
|
|
<pre><code>mlr --csv --json cat data.csv > data.json</code></pre>
|
|
<p>Convert CSV to JSON.</p>
|
|
<h3>Multiple Operations</h3>
|
|
<pre><code>mlr --csv filter '$age > 18' then cut -f name,email then sort -f name users.csv</code></pre>
|
|
<p>Chain operations with 'then'.</p>
|
|
<hr>
|
|
<h2>Tips</h2>
|
|
<ul>
|
|
<li>Use --csv, --tsv, --json to specify input format</li>
|
|
</ul>
|
|
<ul>
|
|
<li>Chain operations with 'then' keyword</li>
|
|
</ul>
|
|
<ul>
|
|
<li>Use $field to reference fields in expressions</li>
|
|
</ul>
|
|
<ul>
|
|
<li>Supports streaming for large files</li>
|
|
</ul>
|
|
<ul>
|
|
<li>Great for data transformation pipelines</li>
|
|
</ul>
|
|
<ul>
|
|
<li>More intuitive than awk for structured data</li>
|
|
</ul>
|
|
<ul>
|
|
<li>Supports many statistical functions</li>
|
|
</ul>
|
|
<ul>
|
|
<li>Excellent documentation and examples</li>
|
|
</ul>
|
|
</div>
|
|
</div>
|
|
</div>
|
|
</div>
|
|
<script async type="text/javascript" src="../blog/analytics.js"></script>
|
|
<script src="../theme.js"></script>
|
|
</body>
|
|
</html>
|
|
|