目前通过ComputeCapabilitiesFilter 是可以做到cpu 指令集的filter,ComputeCapabilitiesFilter 是通过flavor的extra_spe来传递的。
传递的形式为:capabilities:cpu_info:features=ssse3
实现方式:
1.compute_capabilities_filter.py中首先解析flavor-extra_spec , 得到capabilities:cpu_info:features
if scope[0] != "capabilities":
continue
2.得到host_state中的cpu_info:features
(Pdb) pp host_state.cpu_info
u'{"vendor": "Intel", "model": "SandyBridge", "arch": "x86_64", "features":
["pge", "avx", "clflush", "sep", "syscall", "vme", "dtes64", "msr", "fsgsbase", "xsave", "vmx",
"erms", "xtpr", "cmov", "smep", "ssse3", "est", "pat", "monitor", "smx", "pbe",
"lm", "tsc", "nx", "fxsr", "tm", "sse4.1", "pae", "sse4.2", "pclmuldq", "acpi",
"tsc-deadline", "mmx", "osxsave", "cx8", "mce", "de", "tm2", "ht", "pse", "lahf_lm",
"popcnt", "mca", "apic", "sse", "f16c", "ds", "pni", "rdtscp", "aes", "sse2", "ss", "ds_cpl",
"pcid", "fpu", "cx16", "pse36", "mtrr", "pdcm", "rdrand", "x2apic"],
"topology": {"cores": 2, "cells": 1, "threads": 2, "sockets": 1}}' pp instance_type
Flavor(created_at=None,deleted=False,deleted_at=None,disabled=False,ephemeral_gb=0,
extra_specs={capabilities:cpu_info:arch='x86_64',capabilities:cpu_info:features='<or> acpi,adx,avx,avx2'},
flavorid='',id=2,is_public=True,memory_mb=512,name='m1.tiny',projects=<?>,root_gb=1,rxtx_factor=1.0,swap=0,updated_at=None,vcpu_weight=0,vcpus=1) (Pdb) pp scope
[u'cpu_info', u'features'] (Pdb) pp req
u'<or> acpi,adx,avx,avx2'
3. 最后比较req中cpu的需要是否host_state中的cpu_info:features能满足。
那么,host_state中的cpu_info:features怎么获得?
1.resource_tracker.py: self.compute_node.update_from_virt_driver(resources)
def update_from_virt_driver(self, resources):
# NOTE(pmurray): the virt driver provides a dict of values that
# can be copied into the compute node. The names and representation
# do not exactly match.
# TODO(pmurray): the resources dict should be formalized.
keys = ["vcpus", "memory_mb", "local_gb", "cpu_info",
"vcpus_used", "memory_mb_used", "local_gb_used",
"numa_topology", "hypervisor_type",
"hypervisor_version", "hypervisor_hostname",
"disk_available_least", "host_ip"]
for key in keys:
if key in resources:
self[key] = resources[key]
2.compute_node 把信息report给 scheduler,filter就得到了 scheduler的host_state
Nova中还有一个object VirtCPUModel, 它是 instance object 的一部分,用来描述guest cpu模型,其详细说明在:
https://access.redhat.com/documentation/en-US/Red_Hat_Enterprise_Linux/6/html/Virtualization_Administration_Guide/sect-libvirt-dom-xml-cpu-model-top.html
@base.NovaObjectRegistry.register
class VirtCPUModel(base.NovaObject):
# Version 1.0: Initial version
VERSION = '1.0' fields = {
'arch': fields.ArchitectureField(nullable=True),
'vendor': fields.StringField(nullable=True),
'topology': fields.ObjectField('VirtCPUTopology',
nullable=True),
'features': fields.ListOfObjectsField("VirtCPUFeature",
default=[]),
'mode': fields.CPUModeField(nullable=True),
'model': fields.StringField(nullable=True),
'match': fields.CPUMatchField(nullable=True),
}
object的填充方式:
guest.cpu = self._get_guest_cpu_config(
flavor, image_meta, guest_numa_config.numaconfig,
instance.numa_topology) # Notes(yjiang5): we always sync the instance's vcpu model with
# the corresponding config file.
instance.vcpu_model = self._cpu_config_to_vcpu_model(
guest.cpu, instance.vcpu_model)
mode/model通过nova.conf来配置
def _get_guest_cpu_model_config(self):
mode = CONF.libvirt.cpu_mode
model = CONF.libvirt.cpu_model
topology 通过flave-extra_spec/ image meta
virt.hardware.py
def _get_cpu_topology_constraints(flavor, image_meta):
"""Get the topology constraints declared in flavor or image :param flavor: Flavor object to read extra specs from
:param image_meta: nova.objects.ImageMeta object instance Gets the topology constraints from the configuration defined
in the flavor extra specs or the image metadata. In the flavor
this will look for hw:cpu_sockets - preferred socket count
hw:cpu_cores - preferred core count
hw:cpu_threads - preferred thread count
hw:cpu_max_sockets - maximum socket count
hw:cpu_max_cores - maximum core count
hw:cpu_max_threads - maximum thread count In the image metadata this will look at hw_cpu_sockets - preferred socket count
hw_cpu_cores - preferred core count
hw_cpu_threads - preferred thread count
hw_cpu_max_sockets - maximum socket count
hw_cpu_max_cores - maximum core count
hw_cpu_max_threads - maximum thread count
feature 也是通过flave-extra_spec/ image meta:
class VirtCPUFeature(base.NovaObject):
VERSION = '1.0' fields = {
'policy': fields.CPUFeaturePolicyField(nullable=True),
'name': fields.StringField(nullable=False),
}
features = [objects.VirtCPUFeature(
name=f.name,
policy=f.policy) for f in cpu_config.features]
vcpu_model.features = features
virt.hardware.py
def _add_cpu_pinning_constraint(flavor, image_meta, numa_topology):
flavor_pinning = flavor.get('extra_specs', {}).get("hw:cpu_policy")
image_pinning = image_meta.properties.get("hw_cpu_policy")
实现了cpu instruction 规则force/require/optional/disable/forbid。
通过image来限定有2种:
ImagePropertiesFilter
根据instance's image上定义的属性来过滤,熟悉包括
architecture, hypervisor type, hypervisor version (for Xen hypervisor type only), and virtual machine mode。
其中,他们的值可以为:
arch:
ALL = [
ALPHA,
ARMV6,
ARMV7,
ARMV7B,
AARCH64,
CRIS,
I686,
IA64,
LM32,
M68K,
MICROBLAZE,
MICROBLAZEEL,
MIPS,
MIPSEL,
MIPS64,
MIPS64EL,
OPENRISC,
PARISC,
PARISC64,
PPC,
PPCLE,
PPC64,
PPC64LE,
PPCEMB,
S390,
S390X,
SH4,
SH4EB,
SPARC,
SPARC64,
UNICORE32,
X86_64,
XTENSA,
XTENSAEB,
]
HVType:
ALL = (
BAREMETAL,
BHYVE,
DOCKER,
FAKE,
HYPERV,
IRONIC,
KQEMU,
KVM,
LXC,
LXD,
OPENVZ,
PARALLELS,
PHYP,
QEMU,
TEST,
UML,
VBOX,
VIRTUOZZO,
VMWARE,
XEN,
ZVM,
)
VMMode:
ALL = [HVM, XEN, UML, EXE]
如果image->properties中没有这些属性,则通过过滤;如果有这些属性,而host->capabilities->supportedinstances没有,返回False。
比如,
$ glance image-update img-uuid --property architecture=arm --property hypervisor_type=qemu
2 AggregateImagePropertiesIsolation:
匹配属性定义在 image的元数据中,用于对这些 aggregate 组进行匹配
比如下面这个 aggregate 分组 myWinAgg 将 windows 系统作为元数据 os=windows
$ nova aggregate-details MyWinAgg
+----+----------+-------------------+------------+---------------+
| Id | Name | Availability Zone | Hosts | Metadata |
+----+----------+-------------------+------------+---------------+
| 1 | MyWinAgg | None | 'sf-devel' | 'os=windows' |
+----+----------+-------------------+------------+---------------+
在这个理子中,因为下面 win-2012的镜像中有 windows 的属性, 通过他启动的虚拟机实例会再 sf-devel 分组上启动.
$ glance image-show Win-2012
+------------------+--------------------------------------+
| Property | Value |
+------------------+--------------------------------------+
| Property 'os' | windows |
| checksum | f8a2eeee2dc65b3d9b6e63678955bd83 |
| container_format | ami |
| created_at | 2013-11-14T13:24:25 |
目前社区关于这块的工作:
https://review.openstack.org/#/c/286520/3/specs/newton/approved/expose-host-capabilities.rst
打算扩展和统一host capability,主要在现有的host capability之上扩展以下3个方面:
- 版本capability
- 用在image filter的architecture, hypervisor type, VM mode
- 不易被发现的e.g.: SSD, SR-IOV, fibre channel, etc。
https://review.openstack.org/#/c/309762/2/specs/newton/approved/standardize-capabilities.rst
试图采用枚举的方式(nova.objects.fields.Enum-based)来同一capability的描述模型。
它首先定义了capabilities:
A capability is a *singular* piece of information describing some **functionality** or **feature** of an entity that can be provided to a custom。
比如:
* `hw:x86_cpu_instruction_set_ext:sse`
* `hw:x86_cpu_instruction_set_ext:mmx`
用来表述CPU的指令集
* `virt:libvirt:set_admin_pass`
* `virt:libvirt:huge_pages`
* `virt:libvirt:uefi_boot`
用来表述libvirt的capability.