Openstack Swift学习(六)：服务启动源码分析

2018-09-29

先介绍下 swift 的代码结构：

bin/: Executable scripts that are the processes run by the deployer
doc/: Documentation
etc/: Sample config files
examples/: Config snippets used in the docs
swift/: Core code
- account/: account server
- cli/: code that backs some of the CLI tools in bin/
- common/: code shared by different modules
  - middleware/: “standard”, officially-supported middleware
  - ring/: code implementing Swift’s ring
- container/: container server
- locale/: internationalization (translation) data
- obj/: object server
- proxy/: proxy server
test/: Unit, functional, and probe tests

部署完成 swift 以后，就可以运行类似 swift-init proxy start, swift-init account-server start, swift-init container-server start, swift-init object-server start, swift-init main start, swift-init all start 等命令来启动相应的服务。

swift 的服务分别有：account-auditor, account-server, container-auditor, container-replicator, container-reconciler, container-server, container-sharder, container-sync, container-updater, object-auditor, object-server, object-expirer, object-replicator, object-reconstructor, object-updater, proxy-server, account-replicator, account-reaper

类似 start 的命令有： status, start, force-reload, stop, no-wait, no-daemon, reload, shutdown, restart, once

同时，server 和 start 的位置可以互换，还支持正则匹配符号 *, 如： swift-init proxy start 也可以用 swift-init start proxy 来代替，还可以用 swift-init pro* start。

下面以在 proxy node 上启动 proxy server 为例来介绍，启动其他服务也是类似。

swift-init proxy start 命令后面是可以带参数的，可以用来重写配置文件中的部分配置，相关的参数可以在 bin/swift-init 中的 main 函数看到。

当运行 swift-init proxy start 命令后，首先运行的是 bin/swift-init 中的 main 函数，如下

def main():
    # options为命令行输入的参数，默认值 = {'run_dir': '/var/run/swift', 'daemon': True,
    #            'verbose': False, 'number': 0, 'kill_wait': 15,
    #            'graceful': False, 'wait': True, 'once': False}
    # once：if it's True，父进程等待子进程结束后再往下运行
    options, args = parser.parse_args()

    # 分割输入的命令
    # eg: swift-init proxy start
    # then: command = start, servers = ['proxy']
    command = args[-1]  # 最后一个
    servers = args[:-1] # 第一个到倒数第二个

    # 默认启动方式是： swift-init proxy start，如果输入的是：swift-init start proxy
    # 那么需要将 command 和 servers 中的内容互换
    commands = dict(Manager.list_commands()).keys()
    if command not in commands and servers[0] in commands:
        servers.append(command)
        command = servers.pop(0)

    # 实例化类Manager；
    # Manager是直接管理各个servers的类；
    # 初始化各个服务；
    manager = Manager(servers, run_dir=options.run_dir)
    try:
        status = manager.run_command(command, **options.__dict__)
    except UnknownCommandError:
        parser.print_help()
        print('ERROR: unknown command, %s' % command)
        status = 1
    return 1 if status else 0

进一步看 Manager 类的构造函数，看看是怎么实例化的。其存在于 swift/swift/common/manager.py 中

class Manager(object):
    def __init__(self, servers, run_dir=RUN_DIR):
        self.server_names = set()
        self._default_strict = True
        for server in servers:
            # ALIASES = {'all': ALL_SERVERS, 'main': MAIN_SERVERS, 'rest': REST_SERVERS}
            # ALL_SERVERS, MAIN_SERVERS,REST_SERVERS 有相应的宏定义
            if server in ALIASES:
                self.server_names.update(ALIASES[server])
                self._default_strict = False
            # MAIN_SERVERS = ['proxy-server', 'account-server', 'container-server', 'object-server']
            elif '*' in server:
                # 可以用*匹配server的名称，eg: swift-init pro* start
                self.server_names.update([
                    s for s in ALL_SERVERS if
                    re.match(server.replace('*', '.*'), s)])
                self._default_strict = False
            else:
                self.server_names.add(server)

        self.servers = set()
        # 依次初始化相应的服务并添加到 manager 的 servers 属性中
        for name in self.server_names:
            self.servers.add(Server(name, run_dir))

实例化完 manager 对象后，其 servers 属性中保存了已经实例化了的需要进行操作的服务。然后调用 manager 的 run_command() 函数来对相应的服务进行操作。

class Manager(object):

    def run_command(self, cmd, **kwargs):
        # cmd 是要调用的命令,下面的之一
        # ['status', 'start', 'force-reload', 'stop','no-wait',
        #  'no-daemon', 'reload', 'shutdown', 'restart', 'once']
        f = self.get_command(cmd)
        return f(**kwargs)  # eg: start(**kwargs),stop(**kwargs),...

调用 run_command() 后会根据在命令行输入的命令类型(如：start,restart,stop,…)来决定调用 manager 的哪个函数来对 server 进行相应的操作。

如果命令行输入的是 swift-init proxy start，那么则会在 run_command() 中调用 start() 函数。

class Manager(object):

    def start(self, **kwargs):

        setup_env()
        status = 0

        # kwargs是 swift-init:main() 中 options 传过来的参数
        strict = kwargs.get('strict')
        # if strict not set explicitly
        if strict is None:
            strict = self._default_strict

        # 依次调用 server.launch() 启动服务
        for server in self.servers:
            status += 0 if server.launch(**kwargs) else 1

        if not strict:
            status = 0

        if not kwargs.get('daemon', True):
            for server in self.servers:
                try:
                    status += server.interact(**kwargs)
                except KeyboardInterrupt:
                    print(_('\nuser quit'))
                    self.stop(**kwargs)
                    break
        elif kwargs.get('wait', True):
            for server in self.servers:
                status += server.wait(**kwargs)
        return status

我们进一步看 Server 类的 launch() 函数是如何启动一个服务的。

class Server(object):

    def launch(self, **kwargs):
        conf_files = self.conf_files(**kwargs)
        if not conf_files:
            return {}

        # 获取正在运行的进程,返回一个 {pids:pid_files} 的字典
        # 如果没有进程在运行则返回空
        pids = self.get_running_pids(**kwargs)

        # pids 不为空则表示已经有进程在运行
        # 下面判断服务是否启动
        already_started = False
        for pid, pid_file in pids.items():
            conf_file = self.get_conf_file_name(pid_file)
            # for legacy compat you can't start other servers if one server is
            # already running (unless -n specifies which one you want), this
            # restriction could potentially be lifted, and launch could start
            # any unstarted instances
            if conf_file in conf_files:
                already_started = True
                print(_("%(server)s running (%(pid)s - %(conf)s)") %
                      {'server': self.server, 'pid': pid, 'conf': conf_file})
            elif not kwargs.get('number', 0):
                already_started = True
                print(_("%(server)s running (%(pid)s - %(pid_file)s)") %
                      {'server': self.server, 'pid': pid,
                       'pid_file': pid_file})

        if already_started:
            print(_("%s already started...") % self.server)
            return {}

        # START_ONCE_SERVERS 表示在 ALL_SERVERS 中，但不在 MAIN_SERVERS
        # 即，除 'proxy-server', 'account-server', 'container-server',
        #       'object-server' 之外的所有服务
        if self.server not in START_ONCE_SERVERS:
            kwargs['once'] = False

        pids = {}
        for conf_file in conf_files:
            if kwargs.get('once'):
                msg = _('Running %s once') % self.server
            else:
                msg = _('Starting %s') % self.server
            print('%s...(%s)' % (msg, conf_file))
            try:
                # 启动服务
                pid = self.spawn(conf_file, **kwargs)
            except OSError as e:
                if e.errno == errno.ENOENT:
                    # TODO(clayg): should I check if self.cmd exists earlier?
                    print(_("%s does not exist") % self.cmd)
                    break
                else:
                    raise
            pids[pid] = conf_file

        return pids

swift 中各个 server 的进程 pid 都是写入到文件中，默认保存在路径 /var/run/swift 下，我们看到上面 launch() 先查看指定路径下是否存在相应的文件，从而判断相应的服务是否已经启动，如果没有启动再调用 spawn() 启动。下面进一步看 spawn() 函数。

class Server(object):
    def spawn(self, conf_file, once=False, wait=True, daemon=True,
              additional_args=None, **kwargs):
        """Launch a subprocess for this server.

        :param conf_file: path to conf_file to use as first arg
        :param once: boolean, add once argument to command
        :param wait: boolean, if true capture stdout with a pipe
        :param daemon: boolean, if false ask server to log to console
        :param additional_args: list of additional arguments to pass
                                on the command line

        :returns: the pid of the spawned process
        """
        args = [self.cmd, conf_file]
        if once:
            args.append('once')
        if not daemon:
            # ask the server to log to console
            args.append('verbose')
        if additional_args:
            if isinstance(additional_args, str):
                additional_args = [additional_args]
            args.extend(additional_args)

        # figure out what we're going to do with stdio
        if not daemon:
            # do nothing, this process is open until the spawns close anyway
            re_out = None
            re_err = None
        else:
            re_err = subprocess.STDOUT
            if wait:
                # we're going to need to block on this...
                re_out = subprocess.PIPE
            else:
                re_out = open(os.devnull, 'w+b')
        # 创建并返回一个子进程，args[0],可执行文件名，args[1:]是参数
        # args = ['swift-**-server', '/etc/swift/**-server.conf']
        proc = subprocess.Popen(args, stdout=re_out, stderr=re_err)
        # 获取 pid_file 路径，eg: /var/run/swift/***-server.pid
        pid_file = self.get_pid_file_name(conf_file)
        # 将 proc.pid 写入到 pid_file
        write_file(pid_file, proc.pid)
        self.procs.append(proc)
        return proc.pid

至此大致流程已经分析完毕

Reference

https://blog.csdn.net/gaoxingnengjisuan/article/details/9285031

https://blog.csdn.net/gaoxingnengjisuan/article/details/9305219

https://blog.csdn.net/gaoxingnengjisuan/article/details/9315627