1. PyInstaller - 打包Python程序
$ pip install pyinstaller
$ pyinstaller yourprogram.py
image.png
2. Locust - 测试工具，可完成压力测试、功能测试等
https://docs.locust.io/en/stable/quickstart.html
- test script
from locust import HttpLocust, TaskSet, task
class UserBehavior(TaskSet):
def on_start(self):
""" on_start is called when a Locust start before any task is scheduled """
self.login()
def on_stop(self):
""" on_stop is called when the TaskSet is stopping """
self.logout()
def login(self):
self.client.post("/login", {"username":"ellen_key", "password":"education"})
def logout(self):
self.client.post("/logout", {"username":"ellen_key", "password":"education"})
@task(2)
def index(self):
self.client.get("/")
@task(1)
def profile(self):
self.client.get("/profile")
class WebsiteUser(HttpLocust):
task_set = UserBehavior
min_wait = 5000
max_wait = 9000
- run for test
$ locust -f locust_files/my_locust_file.py --host=http://example.com
- open http://127.0.0.1:8089 to start and analyse test results
image.png
`
3. luigi - 任务调度系统
3.1. 概述
Luigi is a Python (2.7, 3.6, 3.7 tested) package that helps you build complex pipelines of batch jobs. It handles dependency resolution, workflow management, visualization, handling failures, command line integration, and much more.
image.png
image.png
3.2. demo
3.2.1. hello world
- hello_world.py
import luigi
class HelloWorldTask(luigi.Task):
task_namespace = 'examples'
def run(self):
print("{task} says: Hello world!".format(task=self.__class__.__name__))
if __name__ == "__main__":
luigi.run(['examples.HelloWorldTask', '--workers', '1', '--local-scheduler'])
- run
$ python hello_world.py
...
===== Luigi Execution Summary =====
Scheduled 1 tasks of which:
* 1 ran successfully:
- 1 examples.HelloWorldTask()
This progress looks :) because there were no failed tasks or missing dependencies
===== Luigi Execution Summary =====
3.2.2. foo-bar
- foo.py
from __future__ import print_function
import time
import luigi
class Foo(luigi.WrapperTask):
task_namespace = 'examples'
def run(self):
print("Running Foo")
def requires(self):
for i in range(10):
yield Bar(i)
class Bar(luigi.Task):
task_namespace = 'examples'
num = luigi.IntParameter()
def run(self):
time.sleep(1)
self.output().open('w').close()
def output(self):
"""
Returns the target output for this task.
:return: the target output for this task.
:rtype: object (:py:class:`~luigi.target.Target`)
"""
time.sleep(1)
return luigi.LocalTarget('/tmp/bar/%d' % self.num)
if __name__ == '__main__':
luigi.run()
- run
$ python foo.py examples.Foo
...
===== Luigi Execution Summary =====
Scheduled 1 tasks of which:
* 1 complete ones were encountered:
- 1 examples.Foo()
Did not run any tasks
This progress looks :) because there were no failed tasks or missing dependencies
===== Luigi Execution Summary =====
3.3. 参考
luigi examples
https://github.com/spotify/luigi/tree/master/examplesluigi documents
https://luigi.readthedocs.io/en/stable/tasks.html浅谈工作流调度系统
https://blog.csdn.net/hsg77/article/details/53409076
