电脑/python/第三方模块/luigi框架.html

<!DOCTYPE html>
<html>
<head>
<!-- 2016-07-30 六 23:04 -->
<meta  charset="utf-8">
<meta  name="viewport" content="width=device-width, initial-scale=1">
<title>luigi框架</title>
<meta  name="generator" content="Org-mode">
<meta  name="author" content="万泽(德山书生)">
<meta  name="description" content="制作者邮箱：a358003542@gmail.com"
>
<style type="text/css">
body {
    margin-right: auto;
    margin-left: auto;
}

.title {
    display: block;
    text-align: center;
    padding: 10px;
}

.center-block {
    display: block;
    margin-left: auto;
    margin-right: auto;
}

.underline{
    text-decoration: underline;
}
   html, body {
    margin: 0;
    padding: 0;
    font-family:"Microsoft YaHei",Arial,Helvetica,sans-serif,"宋体";
    font-size: 1em;
}

#content {
    display: block;
    margin-left: auto;
    margin-right: auto;
    background:#fcfcfc;
    height:100%;
    margin-left:300px;
    max-width:800px;
    min-height:100%;
    padding:1.618em 3.236em;
}

code {
    padding: 2px 4px;
    color: #c7254e;
    background-color: #f9f2f4;
    border-radius: 4px;
}

pre {
    max-width: 100%;
    display: block;
    padding: 9.5px;
    margin: 0 0 10px;
    font-size: 13px;
    line-height: 1.42857143;
    color: #333;
    background-color: #f5f5f5;
    border: 1px solid #ccc;
    border-radius: 4px;
    overflow : auto;
}


blockquote {
    padding: 10px 20px;
    margin: 0 0 20px;
    font-size: 17.5px;
    border-left: 5px solid #eee;
}

li{
    margin-top: 0.5em;
    margin-bottom: 0.5em;
}

p{
    text-indent:2em;
    line-height:150%;
}

/* dl */
dl {
    margin-top: 0;
    margin-bottom: 20px;
}
dt {
    font-weight: 700;
    line-height: 150%;
    margin-top: 0.5em;
    margin-bottom: 0.5em;
}
dd {
    line-height: 150%;
}

@media (min-width: 768px) {
  dt {
    white-space: nowrap;
  }
}

video{
    max-width: 100%;
    margin-left: auto;
    margin-right: auto;
}

figure p{
    text-indent:0em;
}
img{
    max-width: 100%;
}

embed{
    max-width: 100%;
    margin-left: auto;
    margin-right: auto;
}

figure{
    text-align: center;
}

/*  class  */
.FRAMED{
    max-width:100%;
    border:1px solid ;
    padding: 1em;
    -webkit-box-sizing: border-box;
    -moz-box-sizing: border-box;
    box-sizing: border-box;
}

.NOTECARD{
    width: 35%;
    position:relative;
    right: -30%;
    padding: 1em;
    margin:0 auto;
    border: solid 1px;
    -webkit-box-sizing: border-box;
    -moz-box-sizing: border-box;
    box-sizing: border-box;
}

/*
table
*/
table {
    border-collapse: collapse;
    border-spacing: 0;
    margin: 16px 0;
    empty-cells: show;
    border: 1px solid #ccc;
    width: 100%;
    display: table;
}

table tr {
    border-bottom: 1px solid #ddd;
}
table th,table td{
    padding:8px;
}

table tr:nth-child(even) {
    background-color: #f1f1f1;
}


ul,ol,dl{
    line-height:24px;
    list-style-image:none;
    /* list-style:none; */
    margin:0px 0px 24px 0px;
    padding:0;
}

li{
    margin-left: 24px;
}

dd{
    margin:0;
}

#content .section ul,#content .toctree-wrapper ul,article ul{
    list-style:disc;
    line-height:24px;
    margin-bottom:24px}

#content .section ul li,#content .toctree-wrapper ul li,article ul li{
    list-style:disc;
    margin-left:24px}

#content .section ul li p:last-child,#content .toctree-wrapper ul li p:last-child,article ul li p:last-child{
                                                                                                                                  margin-bottom:0}

#content .section ul li ul,#content .toctree-wrapper ul li ul,article ul li ul{
    margin-bottom:0}

#content .section ul li li,#content .toctree-wrapper ul li li,article ul li li{
    list-style:circle}

#content .section ul li li li,#content .toctree-wrapper ul li li li,article ul li li li{
    list-style:square}

#content .section ul li ol li,#content .toctree-wrapper ul li ol li,article ul li ol li{
    list-style:decimal}

#content .section ol,#content ol,article ol{
    list-style:decimal;
    line-height:24px;
    margin-bottom:24px}

#content .section ol li,#content ol li,article ol li{
    list-style:decimal;
    margin-left:24px}

#content .section ol li p:last-child,#content ol li p:last-child,article ol li p:last-child{
                                                                                                                           margin-bottom:0}

#content .section ol li ul,#content ol li ul,article ol li ul{
    margin-bottom:0}

#content .section ol li ul li,#content ol li ul li,article ol li ul li{
    list-style:disc}

dl dt{
    font-weight:bold;
}

dl p,dl table,dl ul,dl ol{
    margin-bottom:12px !important;
}

dl dd{
    margin:0 0 12px 24px;
}

@media print{
    .codeblock,pre.src{
        white-space:pre.src-wrap}
}

@media print{
    html,body,section{
        background:none !important}

    *{
        box-shadow:none !important;
        text-shadow:none !important;
        filter:none !important;
        -ms-filter:none !important}

    a,a:visited{
          text-decoration:underline}

    pre.src,blockquote{
        page-break-inside:avoid}

    thead{
        display:table-header-group}

    tr,img{
        page-break-inside:avoid}

    img{
        max-width:100% !important}

    @page{
        margin:0.5cm}

    p,h2,h3{
        orphans:3;
        widows:3}

    h2,h3{
        page-break-after:avoid}
}

@media print{
    #postamble{
        display:none}

    #content{
        margin-left:0}
}

@media print{
    #table-of-contents{
        display:none}

    @page{
        size: auto;
        margin: 25mm 25mm 25mm 25mm;}

    body {
        margin: 0px;}
}

@media screen and (max-width: 768px){
}

@media only screen and (max-width: 480px){
}

@media screen and (max-width: 768px){
    .tablet-hide{
        display:none}
}

@media screen and (max-width: 480px){
    .mobile-hide{
        display:none}
}

@media screen and (max-width: 480px){
}

@media screen and (max-width: 768px){
    #content{
        margin-left:0;
        padding: 1em 1em;}

    #content #content{
        padding:1.618em}

    #content.shift{
        position:fixed;
        min-width:100%;
        left:85%;
        top:0;
        height:100%;
        overflow:hidden}
}

@media screen and (min-width: 1400px){
    #content{
        background:rgba(0,0,0,0.05)}

    #content{
        background:#fcfcfc}
}

@media screen and (max-width: 768px){
    #copyright{
        width:85%;
        display:none}

    #copyright.shift{
        display:block}

    img{
        width:100%;
        height:auto}
}

@media screen and (max-width: 480px){
    #content .sidebar{
        width:100%}
}


.figure p{
    color:#000;
    font:italic 85%/1 arial,sans-serif;
    padding:1em 0;
}

.rotate-90{
    -webkit-transform:rotate(90deg);
    -moz-transform:rotate(90deg);
    -ms-transform:rotate(90deg);
    -o-transform:rotate(90deg);
    transform:rotate(90deg);
}

.rotate-270{
    -webkit-transform:rotate(270deg);
    -moz-transform:rotate(270deg);
    -ms-transform:rotate(270deg);
    -o-transform:rotate(270deg);
    transform:rotate(270deg);
}

#toggle-sidebar,
#table-of-contents .close-sidebar {
    display: none;
}

@media screen and (max-width: 768px) {
    #table-of-contents {
        display: none;
        width: 60%;
    }

    #table-of-contents h2 a {
        display: block;
    }

    #table-of-contents:target {
        display: block;
    }

    #copyright, #postamble {
        display: none;
    }

    #toggle-sidebar {
        background-color: #2980B9;
        display: block;
        margin-bottom: 1.6em;
        padding: 0.6em;
        text-align: center;
    }

    #toggle-sidebar h2 {
        color: white;
        font-size: 100%;
        line-height: 50px;
        margin: 0;
        padding: 0;
    }

    #table-of-contents .close-sidebar {
       color: rgba(255, 255, 255, 0.3);
       display: inline-block;
       margin: 0px 10px 0px 45px;
       padding: 10px;
    }
}

*{
    -webkit-box-sizing:border-box;
    -moz-box-sizing:border-box;
    box-sizing:border-box;
}

figcaption,figure,footer,header,hgroup,nav{
    display:block}

ins{
    background:#ff9;
    color:#000;
    text-decoration:none}

mark{
    background:#ff0;
    color:#000;
    font-style:italic;
    font-weight:bold}

small{
    font-size:85%}

sub,sup{
    font-size:75%;
    line-height:0;
    position:relative;
    vertical-align:baseline}

sup{
    top:-0.5em}

sub{
    bottom:-0.25em}

img{
    -ms-interpolation-mode:bicubic;
    vertical-align:middle;
    max-width:100%}

svg:not(:root){
    overflow:hidden}

figure{
    margin:0}

label{
    cursor:pointer}

legend{
    border:0;
    margin-left:-7px;
    padding:0;
    white-space:normal}

#content .danger,#content .error{
    background:#fdf3f2}

legend{
    display:block;
    width:100%;
    border:0;
    padding:0;
    white-space:normal;
    margin-bottom:24px;
    font-size:150%;
    *margin-left:-7px}

label{
    display:block;
    margin:0 0 0.3125em 0;
    color:#333;
    font-size:90%}

a{
    color:#2980B9;
    text-decoration:none;
    cursor:pointer}


a:hover,a:active{
    outline:0;
}

a:hover{
    color:#3091d1}

a:visited{
    color:#9B59B6}

.left{
    text-align:left}

.center{
    text-align:center}

.right{
    text-align:right}

hr{
    display:block;
    height:1px;
    border:0;
    border-top:1px solid #e1e4e5;
    margin:24px 0;
    padding:0}

#table-of-contents li{
    list-style:none;
    margin-left: 0px;
}

#table-of-contents header{
    height:32px;
    display:inline-block;
    line-height:32px;
    padding:0 1.618em;
    display:block;
    font-weight:bold;
    text-transform:uppercase;
    font-size:80%;
    color:#2980B9;
    white-space:nowrap}

#table-of-contents ul{
    margin-bottom:0}

#table-of-contents li.divide-top{
    border-top:solid 1px #404040}

#table-of-contents li.divide-bottom{
    border-bottom:solid 1px #404040}

#table-of-contents li.current{
    background:#e3e3e3}

#table-of-contents li.current a{
    color:gray;
    border-right:solid 1px #c9c9c9;
    padding:0.4045em 2.427em}

#table-of-contents li.current a:hover{
    background:#d6d6d6}

#table-of-contents li a{
    /* color:#404040; */
    padding:0.4045em 1.618em;
    position:relative;
    /* background:#fcfcfc; */
    border:none;
    /* border-bottom:solid 1px #c9c9c9; */
    /* border-top:solid 1px #c9c9c9; */
    padding-left:1.618em -4px}

#table-of-contents li.on a:hover,#table-of-contents li.current>a:hover{
    background:#fcfcfc}

#table-of-contents li ul li a{
    /* background:#c9c9c9; */
    padding:0.4045em 2.427em}

#table-of-contents li ul li ul li a{
    padding:0.4045em 3.236em}

#table-of-contents li.current ul{
    display:block}

/* #table-of-contents li ul{ */
/*     margin-bottom:0; */
/*     display:none} */

#table-of-contents .local-toc li ul{
    display:block}

#table-of-contents li ul li a{
    margin-bottom:0;
    color:#b3b3b3;
    font-weight:normal}

#table-of-contents a{
    display:inline-block;
    line-height:18px;
    padding:0.4045em 1.618em;
    display:block;
    position:relative;
    font-size:90%;
    color:#b3b3b3;
    direction: ltr;
}

#table-of-contents a:hover{
    background-color:#4e4a4a;
    cursor:pointer}

/* #text-table-of-contents { */
/*     overflow:scroll; */
/* } */

#table-of-contents{
    position:fixed;
    top:0;
    left:0;
    width:300px;
    overflow-x:hidden;
    overflow-y:scroll;
    height:100%;
    background:#343131;
    z-index:200;
    scrollbar-base-color: #1F1D1D;
    scrollbar-arrow-color: #b3b3b3;
    scrollbar-shadow-color: #1F1D1D;
    scrollbar-track-color : #343131;
}

#table-of-contents h2{
    z-index:200;
    background-color:#2980B9;
    text-align:center;
    padding:0.809em;
    display:block;
    color:#fcfcfc;
    font-size: 100%;
    margin-bottom:0.809em}

ul.nav li ul li {
    display: none;
}

ul.nav li ul li ul li {
    display: none;
}

ul.nav li.active ul li {
    display: inline;
}

ul.nav li.active ul li ul li {
    display: inline;
}

ul.nav li.active ul li a {
    background-color: #E3E3E3;
    color: #8099B0;
    border-right:solid 1px #c9c9c9 !important;
}

ul.nav li.active ul li.active a {
    background-color: #C9C9C9;
    color: black !important;
    font-weight: bold !important;
}

ul.nav li.active ul li.active ul li.active a {
    color: black !important;
    font-weight: bold !important;
    display: block !important;
}

ul.nav li.active ul li.active ul li a {
    color: #808080 !important;
    font-weight: normal !important;
    display: block !important;
}

ul.nav li.active ul li ul li a {
    display: none !important;
}


ul.nav li ul li ul li ul li {
    display: none !important; /* as long as nav is on multiple levels of ul */
    /* display: none; /* as long as nav is on multiple levels of ul *\/ */
}

ul.nav li.active > a {
    border-bottom:solid 1px #c9c9c9 !important; /* XXX Restrict it to 2nd level */
    border-right:solid 1px #c9c9c9 !important;
}

ul.nav li.active a {
    color: gray !important;
    font-weight:bold;
    background-color: white;
    border-right:solid 0px white !important;
}

ul.nav > li.active > a {
    color: black !important;
}

footer{
    color:#999}

footer p{
    margin-bottom:12px}

#copyright, #postamble{
    position:fixed;
    bottom:0;
    left:0;
    width:300px;
    color:#fcfcfc;
    background:#1f1d1d;
    border-top:solid 10px #343131;
    font-family:"Lato","proxima-nova","Helvetica Neue",Arial,sans-serif;
    font-size: 90%;
    z-index:400;
    padding:12px;
}

#postamble .author {
    font-size: 100%;
    margin-bottom: 0px;
}

#postamble .date {
    font-size: 90%;
    margin-bottom: 0px;
    color: #27AE60;
}

#postamble .creator,#postamble .validation {
    display:none;
}

#copyright a{
    color:#2980B9;
    text-decoration:none}

#copyright .rst-current-version{
    padding:12px;
    background-color:#272525;
    display:block;
    text-align:right;
    font-size:90%;
    cursor:pointer;
    color:#27AE60;
    *zoom:1}

#content img{
    max-width:100%;
    height:auto !important}

#content div.figure{
    margin-bottom:24px}

#content div.figure.align-center{
    text-align:center}

#content .section>img,#content .section>a>img{
    margin-bottom:24px}

.verse{
    border-left:5px solid #6AB0DE;
    background-color: #E7F2FA;
    padding: 6px 20px;
    font-style:italic;
}

#content .note .last,#content .attention .last,#content .caution .last,#content .danger .last,#content .error .last,#content .hint .last,#content .important .last,#content .tip .last,#content .warning .last,#content .seealso .last,#content .admonitiontodo .last{
    margin-bottom:0}

#content .admonition-title:before{
    margin-right:4px}

#content .section ol p,#content .section ul p{
    margin-bottom:12px}

#content h1 .headerlink,#content h2 .headerlink,#content h3 .headerlink,#content h4 .headerlink,#content h5 .headerlink,#content h6 .headerlink,#content dl dt .headerlink{
    display:none;
    visibility:hidden;
    font-size:14px}

#content h1 .headerlink:after,#content h2 .headerlink:after,#content h3 .headerlink:after,#content h4 .headerlink:after,#content h5 .headerlink:after,#content h6 .headerlink:after,#content dl dt .headerlink:after{
    visibility:visible;
    content:"";
    font-family:FontAwesome;
    display:inline-block}

#content h1:hover .headerlink,#content h2:hover .headerlink,#content h3:hover .headerlink,#content h4:hover .headerlink,#content h5:hover .headerlink,#content h6:hover .headerlink,#content dl dt:hover .headerlink{
    display:inline-block}

#content .sidebar{
    float:right;
    width:40%;
    display:block;
    margin:0 0 24px 24px;
    padding:24px;
    background:#f3f6f6;
    border:solid 1px #e1e4e5}

#content .sidebar p,#content .sidebar ul,#content .sidebar dl{
    font-size:90%}

#content .sidebar .last{
    margin-bottom:0}

#content .sidebar .sidebar-title{
    display:block;
    font-family:"Roboto Slab","ff-tisa-web-pro","Georgia",Arial,sans-serif;
    font-weight:bold;
    background:#e1e4e5;
    padding:6px 12px;
    margin:-24px;
    margin-bottom:24px;
    font-size:100%}

#content .footnote-reference,#content .citation-reference{
    vertical-align:super;
    font-size:90%}

span[id*='MathJax-Span']{
    color:#404040}

.math{
    text-align:center}

#footnotes{
    border-top:1px solid #e1e4e5;
    padding-top: 36px;
}

h2.footnotes{
    display:none;
}


.footnum:before, .footref:before{
    content:"[";
}

.footnum:after, .footref:after{
    content:"]";
}

.footpara {
    color: #999;
    font-size: 90%;
    font-family:"Lato","proxima-nova","Helvetica Neue",Arial,sans-serif;
    padding-bottom: 8px;
    padding-left: 16px;
    padding-right: 16px;
    padding-top: 8px;
    line-height: 1.25em;
    /* display: inline; */
}
.highlight .hll { background-color: #ffffcc }
.highlight  { background: #f8f8f8; }
.highlight .c { color: #8f5902; font-style: italic } /* Comment */
.highlight .err { color: #a40000; border: 1px solid #ef2929 } /* Error */
.highlight .g { color: #000000 } /* Generic */
.highlight .k { color: #204a87; font-weight: bold } /* Keyword */
.highlight .l { color: #000000 } /* Literal */
.highlight .n { color: #000000 } /* Name */
.highlight .o { color: #ce5c00; font-weight: bold } /* Operator */
.highlight .x { color: #000000 } /* Other */
.highlight .p { color: #000000; font-weight: bold } /* Punctuation */
.highlight .ch { color: #8f5902; font-style: italic } /* Comment.Hashbang */
.highlight .cm { color: #8f5902; font-style: italic } /* Comment.Multiline */
.highlight .cp { color: #8f5902; font-style: italic } /* Comment.Preproc */
.highlight .cpf { color: #8f5902; font-style: italic } /* Comment.PreprocFile */
.highlight .c1 { color: #8f5902; font-style: italic } /* Comment.Single */
.highlight .cs { color: #8f5902; font-style: italic } /* Comment.Special */
.highlight .gd { color: #a40000 } /* Generic.Deleted */
.highlight .ge { color: #000000; font-style: italic } /* Generic.Emph */
.highlight .gr { color: #ef2929 } /* Generic.Error */
.highlight .gh { color: #000080; font-weight: bold } /* Generic.Heading */
.highlight .gi { color: #00A000 } /* Generic.Inserted */
.highlight .go { color: #000000; font-style: italic } /* Generic.Output */
.highlight .gp { color: #8f5902 } /* Generic.Prompt */
.highlight .gs { color: #000000; font-weight: bold } /* Generic.Strong */
.highlight .gu { color: #800080; font-weight: bold } /* Generic.Subheading */
.highlight .gt { color: #a40000; font-weight: bold } /* Generic.Traceback */
.highlight .kc { color: #204a87; font-weight: bold } /* Keyword.Constant */
.highlight .kd { color: #204a87; font-weight: bold } /* Keyword.Declaration */
.highlight .kn { color: #204a87; font-weight: bold } /* Keyword.Namespace */
.highlight .kp { color: #204a87; font-weight: bold } /* Keyword.Pseudo */
.highlight .kr { color: #204a87; font-weight: bold } /* Keyword.Reserved */
.highlight .kt { color: #204a87; font-weight: bold } /* Keyword.Type */
.highlight .ld { color: #000000 } /* Literal.Date */
.highlight .m { color: #0000cf; font-weight: bold } /* Literal.Number */
.highlight .s { color: #4e9a06 } /* Literal.String */
.highlight .na { color: #c4a000 } /* Name.Attribute */
.highlight .nb { color: #204a87 } /* Name.Builtin */
.highlight .nc { color: #000000 } /* Name.Class */
.highlight .no { color: #000000 } /* Name.Constant */
.highlight .nd { color: #5c35cc; font-weight: bold } /* Name.Decorator */
.highlight .ni { color: #ce5c00 } /* Name.Entity */
.highlight .ne { color: #cc0000; font-weight: bold } /* Name.Exception */
.highlight .nf { color: #000000 } /* Name.Function */
.highlight .nl { color: #f57900 } /* Name.Label */
.highlight .nn { color: #000000 } /* Name.Namespace */
.highlight .nx { color: #000000 } /* Name.Other */
.highlight .py { color: #000000 } /* Name.Property */
.highlight .nt { color: #204a87; font-weight: bold } /* Name.Tag */
.highlight .nv { color: #000000 } /* Name.Variable */
.highlight .ow { color: #204a87; font-weight: bold } /* Operator.Word */
.highlight .w { color: #f8f8f8; text-decoration: underline } /* Text.Whitespace */
.highlight .mb { color: #0000cf; font-weight: bold } /* Literal.Number.Bin */
.highlight .mf { color: #0000cf; font-weight: bold } /* Literal.Number.Float */
.highlight .mh { color: #0000cf; font-weight: bold } /* Literal.Number.Hex */
.highlight .mi { color: #0000cf; font-weight: bold } /* Literal.Number.Integer */
.highlight .mo { color: #0000cf; font-weight: bold } /* Literal.Number.Oct */
.highlight .sb { color: #4e9a06 } /* Literal.String.Backtick */
.highlight .sc { color: #4e9a06 } /* Literal.String.Char */
.highlight .sd { color: #8f5902; font-style: italic } /* Literal.String.Doc */
.highlight .s2 { color: #4e9a06 } /* Literal.String.Double */
.highlight .se { color: #4e9a06 } /* Literal.String.Escape */
.highlight .sh { color: #4e9a06 } /* Literal.String.Heredoc */
.highlight .si { color: #4e9a06 } /* Literal.String.Interpol */
.highlight .sx { color: #4e9a06 } /* Literal.String.Other */
.highlight .sr { color: #4e9a06 } /* Literal.String.Regex */
.highlight .s1 { color: #4e9a06 } /* Literal.String.Single */
.highlight .ss { color: #4e9a06 } /* Literal.String.Symbol */
.highlight .bp { color: #3465a4 } /* Name.Builtin.Pseudo */
.highlight .vc { color: #000000 } /* Name.Variable.Class */
.highlight .vg { color: #000000 } /* Name.Variable.Global */
.highlight .vi { color: #000000 } /* Name.Variable.Instance */
.highlight .il { color: #0000cf; font-weight: bold } /* Literal.Number.Integer.Long */
</style>
</head>
<body>
<div id="content">
<header>
<h1 class="title">luigi框架</h1>
</header><nav id="table-of-contents">
<h2>Table of Contents</h2>
<div id="text-table-of-contents">
<ul>
<li><a href="#orgheadline1">1. 简介</a></li>
<li><a href="#orgheadline6">2. task</a>
<ul>
<li><a href="#orgheadline2">2.1. run方法</a></li>
<li><a href="#orgheadline3">2.2. get_params方法</a></li>
<li><a href="#orgheadline4">2.3. output方法</a></li>
<li><a href="#orgheadline5">2.4. requires方法</a></li>
</ul>
</li>
<li><a href="#orgheadline8">3. parameter</a>
<ul>
<li><a href="#orgheadline7">3.1. 参数对象的值</a></li>
</ul>
</li>
<li><a href="#orgheadline9">4. target</a></li>
<li><a href="#orgheadline10">5. 配置文件</a></li>
</ul>
</div>
</nav>


<div class="outline-2">
<h2 id="orgheadline1">简介</h2>
<div class="outline-text-2" id="text-1">
<p>
虽然luigi从源码上很简单，但我觉得最好将其称之为框架。模块和框架的区别在于当你引入一个框架时，整个软件的思路架构都会发生某种变动，而模块并没有那么大的影响，更像是一个额外的函数或者类，方便你的使用。总的来说我是不喜欢框架这种束缚人的东西的，但是顶层流程控制不同，它必然内在地对整个系统的架构发生某种深刻的影响，luigi正是这样一个项目，方便你进行你的软件或系统的顶层流程控制，所以最好称之为框架。
</p>

<p>
luigi框架的github项目地址在 <a href="https://github.com/spotify/luigi">这里</a> ，其可以用pip简单安装之。官方文档也不能说没有或语焉不详，但对于顶层流程控制这个内在极丰富的领域来说，确实文档显得单薄了些。好在luigi框架源码结构清晰，也很简单。不过就算把luigi源码分析得差不多了，至于顶层流程控制这个领域来说，也只能说才刚刚开始。
</p>
</div>
</div>


<div class="outline-2">
<h2 id="orgheadline6">task</h2>
<div class="outline-text-2" id="text-2">
<p>
luigi源码 <code>__init__.py</code> 文件一开始就是
</p>
<pre>
from luigi import task
</pre>

<p>
也确实该框架最核心的一个概念就是这个Task类。下面演示一个最基本的例子，通过这个基本的例子我们能够学到很多东西。
</p>

<div class="highlight"><pre><span></span><span class="kn">import</span> <span class="nn">luigi</span>

<span class="k">class</span> <span class="nc">Test</span><span class="p">(</span><span class="n">luigi</span><span class="o">.</span><span class="n">Task</span><span class="p">):</span>
    <span class="n">count</span> <span class="o">=</span> <span class="n">luigi</span><span class="o">.</span><span class="n">IntParameter</span><span class="p">()</span>
    <span class="n">x</span> <span class="o">=</span> <span class="n">luigi</span><span class="o">.</span><span class="n">Parameter</span><span class="p">(</span><span class="n">default</span><span class="o">=</span><span class="s2">&quot;hello luigi&quot;</span><span class="p">)</span>

    <span class="k">def</span> <span class="nf">run</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
        <span class="k">print</span> <span class="bp">self</span><span class="o">.</span><span class="n">count</span>
        <span class="k">print</span> <span class="bp">self</span><span class="o">.</span><span class="n">x</span>
        <span class="k">print</span> <span class="bp">self</span><span class="o">.</span><span class="n">get_params</span><span class="p">()</span>

<span class="k">if</span> <span class="n">__name__</span> <span class="o">==</span> <span class="s1">&#39;__main__&#39;</span><span class="p">:</span>
    <span class="n">luigi</span><span class="o">.</span><span class="n">run</span><span class="p">()</span>
</pre></div>

<p>
首先我们看到luigi.IntParameter那里，这些变量在Task对象中都可以使用如self.count这样的语法引用，然而更重要的是这些变量都自动进入命令行封装了。
</p>

<p>
读者可以先简单看下这个命令行目前支持的选项，Test类一般为本脚本任务流的最终任务出口:
</p>
<pre>
python test1.py Test --help
</pre>

<p>
其中count参量还需要输入值脚本才能正常运行:
</p>
<pre>
python test1.py Test --local-scheduler --count=10
</pre>

<p>
然后脚本要正常运行你还需要加上 <code>--local-scheduler</code> ，这个所谓的scheduler可以简单的理解为luigi的任务调度器，这里设置为了本地的调度器。luigi还支持远程调度器，这样多台机器上可以使用同一的远程的任务调度器，这些机器会协同来完成任务工作流。
</p>
</div>


<div class="outline-3">
<h3 id="orgheadline2">run方法</h3>
<div class="outline-text-3" id="text-2-1">
<p>
run方法是Task对象很重要的一个方法，表明执行该任务其应该做些什么事情。
</p>
</div>
</div>

<div class="outline-3">
<h3 id="orgheadline3">get_params方法</h3>
<div class="outline-text-3" id="text-2-2">
<p>
Task对象的 <code>get_params</code> 方法（其是一个类方法）会返回本任务对象收集的参数。这在有时需要传递全局型参数时很有用。
</p>
</div>
</div>


<div class="outline-3">
<h3 id="orgheadline4">output方法</h3>
<div class="outline-text-3" id="text-2-3">
<p>
下面看第二个例子:
</p>
<div class="highlight"><pre><span></span><span class="kn">from</span> <span class="nn">__future__</span> <span class="kn">import</span> <span class="n">print_function</span>

<span class="kn">import</span> <span class="nn">luigi</span>

<span class="k">class</span> <span class="nc">Test</span><span class="p">(</span><span class="n">luigi</span><span class="o">.</span><span class="n">Task</span><span class="p">):</span>
    <span class="n">count</span> <span class="o">=</span> <span class="n">luigi</span><span class="o">.</span><span class="n">IntParameter</span><span class="p">()</span>
    <span class="n">x</span> <span class="o">=</span> <span class="n">luigi</span><span class="o">.</span><span class="n">Parameter</span><span class="p">(</span><span class="n">default</span><span class="o">=</span><span class="s2">&quot;hello luigi&quot;</span><span class="p">)</span>


    <span class="k">def</span> <span class="nf">output</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
        <span class="k">return</span> <span class="n">luigi</span><span class="o">.</span><span class="n">LocalTarget</span><span class="p">(</span><span class="s2">&quot;test2out.txt&quot;</span><span class="p">)</span>

    <span class="k">def</span> <span class="nf">run</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
        <span class="k">with</span> <span class="bp">self</span><span class="o">.</span><span class="n">output</span><span class="p">()</span><span class="o">.</span><span class="n">open</span><span class="p">(</span><span class="s2">&quot;w&quot;</span><span class="p">)</span> <span class="k">as</span> <span class="n">f</span><span class="p">:</span>
            <span class="k">print</span><span class="p">(</span><span class="bp">self</span><span class="o">.</span><span class="n">count</span><span class="p">,</span><span class="nb">file</span><span class="o">=</span><span class="n">f</span><span class="p">)</span>
            <span class="k">print</span><span class="p">(</span><span class="bp">self</span><span class="o">.</span><span class="n">x</span><span class="p">,</span><span class="nb">file</span><span class="o">=</span><span class="n">f</span><span class="p">)</span>
            <span class="k">print</span><span class="p">(</span><span class="bp">self</span><span class="o">.</span><span class="n">get_params</span><span class="p">(),</span><span class="nb">file</span><span class="o">=</span><span class="n">f</span><span class="p">)</span>

<span class="k">if</span> <span class="n">__name__</span> <span class="o">==</span> <span class="s1">&#39;__main__&#39;</span><span class="p">:</span>
    <span class="n">luigi</span><span class="o">.</span><span class="n">run</span><span class="p">()</span>
<span class="c1">#python test2.py Test --local-scheduler --count=10</span>
</pre></div>
<p>
这个例子演示了output方法的作用，Test任务对象主体run方法作用和之前一样，这里打开了一个文件，而这个文件对象的获取是借助于output的返回的。这里output方法返回的一定是所谓的Target对象，其一定有一个exists方法，通常对应的是某个文件，如果存在则返回True，如果不存在则返回False（这个存在与否的信息方便luigi进行流程控制）。然后具体到LocalTarget对象，其就对应的是本地的某个文件对象。其有一个open方法，目前只有两种模式"r"或"w"。然后后面就和操作常规python文件对象一样了。
</p>

<p>
这个小程序简单将之前的那些信息写入到那个文件中去了。
</p>
</div>
</div>


<div class="outline-3">
<h3 id="orgheadline5">requires方法</h3>
<div class="outline-text-3" id="text-2-4">
<p>
到requires方法这里
</p>
</div>
</div>
</div>


<div class="outline-2">
<h2 id="orgheadline8">parameter</h2>
<div class="outline-text-2" id="text-3">
</div><div class="outline-3">
<h3 id="orgheadline7">参数对象的值</h3>
<div class="outline-text-3" id="text-3-1">
<p>
参数对象进来之后还是什么luigi专门的Parameter对象，并不能在python中正常使用，需要类似 <code>p.value</code> 这样把值提出来之后才能变成常规的python值。
</p>
</div>
</div>
</div>

<div class="outline-2">
<h2 id="orgheadline9">target</h2>
</div>


<div class="outline-2">
<h2 id="orgheadline10">配置文件</h2>
<div class="outline-text-2" id="text-5">
<p>
<a href="http://blog.kissdata.com/2014/05/28/lugi.html#luigi">http://blog.kissdata.com/2014/05/28/lugi.html#luigi</a>
</p>
</div>
</div>
</div>
</body>
</html>