http://cloudstack.apt-get.eu/systemvm/4.6/ 模板地址
http://cloudstack.apt-get.eu/centos7/4.6/ 代理及管理地址
http://mirrors.ustc.edu.cn/fedora/epel/7/x86_64/e/ epel源
http://www.xenproject.org/developers/teams/windows-pv-drivers.html pvdrivers for xenserver
http://www.serverwatch.com/server-tutorials/installing-xenserver-tools-on-windows-8-and-windows-server-2012-virtual-machines.html win8下不能安装pv drivers 和xentools的解决方法
概念
系统虚机在CloudStack中扮演着重要的角色，如系统虚机没有正确导入，CloudStack将无法管理二级存储，也就无法完成导入模板、创建新的虚机等操作。
不同的虚拟机实现方案系统虚机模板也是不同的 导入系统虚拟机模板 /usr/share/cloudstack-common/scripts/storage/secondary/cloud-install-sys-tmplt -m /nfs/secondary/ -u http://192.168.1.98/4/systemvm64template-4.4.1-xen.vhd.bz2 -h xenserver -F /usr/share/cloudstack-common/scripts/storage/secondary/cloud-install-sys-tmplt -m /nfs/secondary/ -u http://192.168.1.98/4/systemvm64template-4.4.0-6-kvm.qcow2.bz2 -h kvm -F 服务方案包括计算方案，系统方案，磁盘方案，网络方案
计算方案通俗的说就是服务器配置，比如4核cpu，8G内存，20G硬盘，这里可以定义服务器硬件配置
定义磁盘方案可以在新增虚拟机的时候，选择磁盘大小 Cloudstack基于tomcat提供web服务，默认使用了8080端口。如果你想改用其它端口，可以修改 /etc/tomcat6/server.xml 文件进行配置。
Cloudstack默认安装在 /etc/cloudstack/management 目录下，你可以通过修改 log4j-cloud.xml文件来调整日志的输出级别、路径等。
cloudstack默认日志在/var/log/cloudstack下
xenserver6.2+cs4.4.2问题
http://ask.cloudstack-china.org/question/111
http://support.citrix.com/article/CTX126531
http://shankerbalan.net/blog/cloudstack-and-xenserver-vhd-util/
问题已经解决！
是xenserver里面的有些设置的问题。我照着下面的做了之后，host就up了：
通过 ssh 登录到 xenserver 主机 1、调整DOM0内存值
//备份配置文件
# cp /boot/extlinux.conf /boot/extlinux.conf.bak20160324
修改dom0_mem值为2940M或者更大(最大不超过4G)
# sed -i '10s/dom0_mem=752M/dom0_mem=2940M/g' /boot/extlinux.conf
修改完成之后，重启系统
# reboot
修改文件权限为722
# chmod 722 /etc/xensource-inventory
2、如果该 主机所在的 CloudStack 的 zone 使用 的是基本网络，则需要禁用OpenVswitch(OVS)
# xe-switch-network-backend bridge
出现 You *MUST* now reboot your system 表示完成
重新启动后OK 如果你发现注册ISO或注册模板时，状态字段一直不动，已就绪永远都是no，那一般都是因为二级存储有问题或Secondary Storage VM 有问题了。
检查二级存储容量是否正确。
检查系统VM中的Secondary Storage VM是否正常启动。 CloudStack不能添加主存储或二级存储
检查/etc/sysconfig/nfs配置文件是否把端口都开放了。
检查iptables是否有阻挡。
检查CloudStack的“全局设置”，secstorage.allowed.internal.sites属性是否设置正确。可以简单地允许一个网段192.168.1.0/24 如果要重来，只需要下面步骤就可以了，清库 298 service cloudstack-management stop
299 cloudstack-setup-databases cloud:password@localhost --deploy-as=root:123456
300 cloudstack-setup-management
301 service cloudstack-management start
302 service iptables stop
307 /usr/share/cloudstack-common/scripts/storage/secondary/cloud-install-sys-tmplt -m /nfs/secondary/ -u http://192.168.1.98/4/systemvm64template-4.4.1-xen.vhd.bz2 -h xenserver -F
删除所有的一级与二级中的相关文件及目录，清空环境，有利于简化问题。
然后再将xenserver上的相关虚拟磁盘全删除，删除pool,再重新创建pool,将主机添加进来。
检查CloudStack的“全局设置”，secstorage.allowed.internal.sites属性是否设置正确。可以简单地允许一个网段192.168.1.0/24,这步是用来注册template/iso用的
报错信息1：
Installing : cloudstack-management-4.6.0-1.el6.x86_64 137/146
Please download vhd-util from http://download.cloud.com.s3.amazonaws.com/tools/vhd-util and put it in
/usr/share/cloudstack-common/scripts/vm/hypervisor/xenserver/
Unable to determine ssl settings for server.xml, please run cloudstack-setup-management manually
Unable to determine ssl settings for tomcat.conf, please run cloudstack-setup-management manually
Updating : db4-utils-4.7.25-20.el6_7.x86_64 报错信息2：如果开始把selinux做成permissive，这步就不会报错
[root@localhost ~]# cloudstack-setup-databases cloud:password@localhost --deploy-as=root
Mysql user name:cloud [ OK ]
Mysql user password:****** [ OK ]
Mysql server ip:localhost [ OK ]
Mysql server port:3306 [ OK ]
Mysql root user name:root [ OK ]
Mysql root user password:****** [ OK ]
Checking Cloud database files ... [ OK ]
Checking local machine hostname ... [ OK ]
Checking SELinux setup ... checkSelinux failed with error continuing... SELinux is set to enforcing. There are two options:
1> Set it permissive in /etc/selinux/config, then reboot the machine.
2> Type 'setenforce Permissive' in commandline, after which you can run this program again. We strongly suggest you doing the option 1 that makes sure SELinux goes into permissive after system reboot.
[ OK ]
Detected local IP address as 192.168.1.111, will use as cluster management server node IP[ OK ]
Preparing /etc/cloudstack/management/db.properties [ OK ]
Applying /usr/share/cloudstack-management/setup/create-database.sql [ OK ]
Applying /usr/share/cloudstack-management/setup/create-schema.sql [ OK ]
Applying /usr/share/cloudstack-management/setup/create-database-premium.sql [ OK ]
Applying /usr/share/cloudstack-management/setup/create-schema-premium.sql [ OK ]
Applying /usr/share/cloudstack-management/setup/server-setup.sql [ OK ]
Applying /usr/share/cloudstack-management/setup/templates.sql [ OK ]
Processing encryption ... [ OK ]
Finalizing setup ... [ OK ] CloudStack has successfully initialized database, you can check your database configuration in /etc/cloudstack/management/db.properties 报错信息3
[root@cs-manage management]# cloudstack-setup-management
Starting to configure CloudStack Management Server:
Configure Firewall ... [OK]
Configure CloudStack Management Server ...[Failed]
Cannot find /etc/cloudstack/management/server-nonssl.xml or /etc/cloudstack/management/tomcat6-nonssl.conf, https enable failed
Try to restore your system:
Restore Firewall ... [OK]
Restore CloudStack Management Server ...[OK] [root@cs-manage management]# cloudstack-setup-management --tomcat7
Starting to configure CloudStack Management Server:
Configure Firewall ... [OK]
Configure CloudStack Management Server ...[OK]
CloudStack Management Server setup is Done!
正常信息：
安装cloudstack数据库
[root@manage ~]# cloudstack-setup-databases cloud:password@localhost --deploy-as=root:123456
Mysql user name:cloud [ OK ]
Mysql user password:****** [ OK ]
Mysql server ip:localhost [ OK ]
Mysql server port:3306 [ OK ]
Mysql root user name:root [ OK ]
Mysql root user password:****** [ OK ]
Checking Cloud database files ... [ OK ]
Checking local machine hostname ... [ OK ]
Checking SELinux setup ... [ OK ]
Detected local IP address as 192.168.1.171, will use as cluster management server node IP[ OK ]
Preparing /etc/cloudstack/management/db.properties [ OK ]
Applying /usr/share/cloudstack-management/setup/create-database.sql [ OK ]
Applying /usr/share/cloudstack-management/setup/create-schema.sql [ OK ]
Applying /usr/share/cloudstack-management/setup/create-database-premium.sql [ OK ]
Applying /usr/share/cloudstack-management/setup/create-schema-premium.sql [ OK ]
Applying /usr/share/cloudstack-management/setup/server-setup.sql [ OK ]
Applying /usr/share/cloudstack-management/setup/templates.sql [ OK ]
Applying /usr/share/cloudstack-bridge/setup/cloudbridge_db.sql [ OK ]
Applying /usr/share/cloudstack-bridge/setup/cloudbridge_schema.sql [ OK ]
Applying /usr/share/cloudstack-bridge/setup/cloudbridge_multipart.sql [ OK ]
Applying /usr/share/cloudstack-bridge/setup/cloudbridge_index.sql [ OK ]
Applying /usr/share/cloudstack-bridge/setup/cloudbridge_multipart_alter.sql [ OK ]
Applying /usr/share/cloudstack-bridge/setup/cloudbridge_bucketpolicy.sql [ OK ]
Applying /usr/share/cloudstack-bridge/setup/cloudbridge_policy_alter.sql [ OK ]
Applying /usr/share/cloudstack-bridge/setup/cloudbridge_offering.sql [ OK ]
Applying /usr/share/cloudstack-bridge/setup/cloudbridge_offering_alter.sql [ OK ]
Processing encryption ... [ OK ]
Finalizing setup ... [ OK ] CloudStack has successfully initialized database, you can check your database configuration in /etc/cloudstack/management/db.properties 初始化管理节点
[root@manage ~]# cloudstack-setup-management
Starting to configure CloudStack Management Server:
Configure sudoers ... [OK]
Configure Firewall ... [OK]
Configure CloudStack Management Server ...[OK]
CloudStack Management Server setup is Done!
启动管理节点
[root@manage ~]# service cloudstack-management start
Starting cloudstack-management: [ OK ] 导入系统模板
[root@manage mnt]# /usr/share/cloudstack-common/scripts/storage/secondary/cloud-install-sys-tmplt -m /mnt/secondary/ -u http://192.168.1.98/4/systemvm64template-4.4.0-6-kvm.qcow2.bz2 -h kvm -F
--2016-03-21 12:09:03-- http://192.168.1.98/4/systemvm64template-4.4.0-6-kvm.qcow2.bz2
Connecting to 192.168.1.98:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 303968528 (290M) [application/x-bzip2]
Saving to: “/usr/share/cloudstack-common/scripts/storage/secondary/bae66adf-d2c3-412d-906e-53db2bc232e8.qcow2” 100%[===================================================================================================================================================================================================>] 303,968,528 11.2M/s in 26s 2016-03-21 12:09:29 (11.2 MB/s) - “/usr/share/cloudstack-common/scripts/storage/secondary/bae66adf-d2c3-412d-906e-53db2bc232e8.qcow2” saved [303968528/303968528] Uncompressing to /usr/share/cloudstack-common/scripts/storage/secondary/bae66adf-d2c3-412d-906e-53db2bc232e8.qcow2.tmp (type bz2)...could take a long time
Moving to /mnt/secondary/template/tmpl/1/3///bae66adf-d2c3-412d-906e-53db2bc232e8.qcow2...could take a while
Successfully installed system VM template to /mnt/secondary/template/tmpl/1/3/ 安装usage
[root@manage mnt]# yum install cloudstack-usage
[root@manage mnt]# service cloudstack-usage start
Starting CloudStack Usage Monitor cloudstack-usage [ OK ]
[root@localhost ~]# history
1 vi /etc/sysconfig/network-scripts/ifcfg-eth0
2 service network restart
7 vi /etc/selinux/config
8 chkconfig iptables off
10 ping qq.com
11 vi /etc/hosts
12 hostname --fqdn
14 service network restart
15 hostname --fqdn
16 yum install -y ntp
17 chkconfig ntpd on
18 service ntpd start
20 vi /etc/yum.repos.d/cloudstack.repo
21 yum -y install nfs-utils
22 mkdir /primary
23 mkdir /secondary
25 vi /etc/exports
26 service rpcbind start
27 service nfs start
32 chkconfig rpcbind on
33 chkconfig --list|grep rpc
34 chkconfig nfs on
35 chkconfig --list|grep nfs
37 yum install mysql-server
38 vi /etc/my.cnf
42 chkconfig mysqld on
43 service mysqld start
45 yum install cloudstack-management
53 cloudstack-setup-databases cloud:password@localhost --deploy-as=root
60 cloudstack-setup-management
/usr/share/cloudstack-common/scripts/storage/secondary/cloud-install-sys-tmplt \
-m /secondary \
-u http://cloudstack.apt-get.eu/systemvm/4.6/systemvm64template-4.6.0-kvm.qcow2.bz2 \
-h kvm -F
cloudstack-setup-agent -m 192.168.1.62 -z 1 -p 1 -c 1 -g d3de6722-e034-34fa-8879-c62e656201a8 -a --pubNic=cloudbr0 --prvNic=cloudbr1 --guestNic=cloud0 --hypervisor=kvm
service cloud-management restart
vi /etc/selinux/config
SELINUX=permissive
vi /etc/yum.repo.d/cloudstack
[cloudstack]
name=cloudstack
baseurl=http://cloudstack.apt-get.eu/centos/6/4.6/
enabled=1
gpgcheck=0
vi /etc/exports
/primary *(rw,async,no_root_squash,no_subtree_check)
/secondary *(rw,async,no_root_squash,no_subtree_check)
vi /etc/my.cnf 在[mysqld]中添加如下
innodb_rollback_on_timeout=1
innodb_lock_wait_timeout=600
max_connections=350
log-bin=mysql-bin
binlog-format = 'ROW'
计算节点上安装代理
yum install cloudstack-agent
Transaction Summary
=============================================================================================================================================================================================================================================
Install 132 Package(s)
Upgrade 6 Package(s)
Total download size: 256 M
[root@computer ~]# history
1 ifconfig
2 vi /etc/sysconfig/network-scripts/ifcfg-eth0
3 service network restart
5 vi /etc/selinux/config
6 chkconfig iptables off
7 reboot
8 getenforce
9 ping qq.com
10 vi /etc/hosts
11 hostname --fqdn
12 service network restart
18 hostname --fqdn
23 yum install ntp
24 chkconfig --list|grep ntp
25 chkconfig ntpd on
26 service ntpd start
27 vi /etc/yum.repos.d/cloudstack.repo
28 yum install cloudstack-agent
29 vi /etc/libvirt/qemu.conf
30 vi /etc/libvirt/libvirt.conf
32 vi /etc/sysconfig/libvirtd
33 service libvirtd start
44 libvirtd -d
45 service libvirtd status
46 lsmod|grep kvm
好像还得安装下面的
yum install kvm python-virtinst libvirt tunctl bridge-utils virt-manager qemu-kvm-tools virt-viewer virt-v2v libguestfs-tools
[root@compute network-scripts]# cloudstack-setup-agent
Welcome to the CloudStack Agent Setup:
Please input the Management Server Hostname/IP-Address:[192.168.1.171]
Please input the Zone Id:[default]
Please input the Pod Id:[default]
Please input the Cluster Id:[default]
Please input the Hypervisor type kvm/lxc:[default]kvm
Please choose which network used to create VM:[cloudbr0]
Starting to configure your system:
Configure Cgroup ... [OK]
Configure SElinux ... [OK]
Configure Network ... [OK]
Configure Libvirt ... [OK]
Configure Firewall ... [OK]
Configure Nfs ... [OK]
Configure cloudAgent ... [OK]
CloudStack Agent setup is done!
vi /etc/libvirt/qemu.conf
vnc_listen=0.0.0.0
vi /etc/libvirt/libvirtd.conf
listen_tls = 0
listen_tcp = 1
tcp_port = "16059"
auth_tcp = "none"
mdns_adv = 0
vi /etc/sysconfig/libvirtd
LIBVIRTD_ARGS="--listen"
公司前不久搭建的cloudstack 4.0(3台XenServer 6.1节点，Centos 6.3管理机器，freenas)由于机房意外停电，造成管理机器无法联系节点，系统vm无法启动，加上时间紧迫，上面有没有上面重要数据，就进行全新安装。安装后，为防止相同事件再次发生，做了简单的灾难恢复测试。 故障模拟
1，系统vm维护
步骤：维护模式关闭系统ssvm cpvm所在物理主机，启用主机维护模式，重启主机
现象：ssvm cpvm 状态处于stop状态
恢复：主机启动完成后，取消维护模式，ssvm cpvm自动启动（5分钟左右）
2，系统vm故障
故障：系统vm丢失
步骤：删除ssvm,cpvm
现象：无法进行添加删除vm操作
恢复：删除的ssvm,cpvm 会自动重建，重建完成后功能恢复
3，系统vrouter故障
故障：系统vrouter丢失
步骤：停止并删除vrouter
现象：客户vm无法通过域名访问，通过ip访问无影响
恢复：重新执行创建vm操作，vrouter自动创建；客户vm需要重启网络服务
4，重启CS管理机器
故障：管理服务器宕机
步骤：直接重启管理机器
现象：无法登陆控制台
恢复：系统启动完成后，正常登陆
5, 存储意外关闭
故障：模拟主存储故障
步骤：重启关闭主存储并禁用网卡10分钟
现象：cpvm,ssvm状态系统正常，cs控制台会有disconneting提示；用户vm状态running,但是无法通过console连接，停止后，无法重启；添加用户vm 一直处于starting状态
恢复：恢复通信后，自动恢复，如果有问题，依次重启各个vm主机
6, 集群主机故障1
故障：所有主机网络故障
步骤：直接关闭主机网络交换机
现象：所有主机状态都为down，系统vm为starting,无法进行用户vm操作
恢复：恢复交换机故障，当master主机通信故障时，及时其他主机通信恢复，其他主机状态依旧为down;master主机恢复通信后，其他主机依次恢复；最后重启用户vm
7, 集群主机故障2
故障：非master主机网络故障
步骤：直接关闭主机网络接口
现象：主机状态都为down，该主机上的用户vm处于shutdown；管理主机日志提示故障主机设置master异常
恢复：直接启动状态为shutdown的用户vm(cloudstack会自动在其他可用主机节点部署用户vm)；恢复故障主机故障后重新加入
8, 集群主机故障3
故障：master主机故障
步骤：直接关闭主机网络接口，模拟意外关闭
现象：主机状态为down，该主机上的用户vm处于shutdown
恢复：其他主机会自动成为master主机;恢复原故障主机通信，仍旧无法加入主机，cs中删除主机，重启;本机登陆命令行，执行xe pool-join 加入到pool,cs中再次添加主机如果需要;资源池的每个成员都将包含担任主主机角色所需的所有信息。 在cloudstack一个cluster中的pool资源池中（通过xenCenter也可以创建资源池），有一个物理主机充当master主机，进行资源的分配；当一个pool中的master主机发生故障，将依次发生下列事件：
1. 成员意识到通信已中断，每个成员都重试 60 秒
2. 然后，各成员将自身置于紧急模式，这样，成员 XenServer 主机现在将仅接受池紧急命令（xe pool-emergency-reset-master 和 xe pool-emergency-transition-to-master）。此时，如果master主机恢复，它将重新建立与其成员的通信，成员退出紧急模式，操作恢复正常。不过，如果master主机已停用，您应该选择一个成员并对其执行 xe pool-emergency-transition-to-master 命令。当该成员成为master主机后，执行命令 xe pool-recover-slaves，其他成员现在将指向此新master主机。如果修复或替换用作原始master主机的服务器，您只需启动服务器，安装 XenServer 主机软件，然后将它添加到池中。由于池中的 XenServer 主机强制为同类主机，因此实际上不需要将替换的服务器设为主主机。将成员 XenServer 主机转换为master主机后，您还应检查默认池存储库是否设置了适当的值。通过使用 xe pool-param-list 命令并验证 default-SR 参数是否指向有效存储库，可实现此操作。 名词解释：xenserver master
在一个xenserver 组成的资源pool中，会选举一个xenserver作为该pool的master,管理工具cloudstack或者xencerter通过操作master,让master决定vm的资源分配情况