activemq负载均衡+高可用方案

发表于 | 分类于

说明

前面有章节介绍了mq的主从复制和集群方案，可以看到Master-Slave的部署方式虽然解决了高可用的问题，但不支持负载均衡，Broker-Cluster解决了负载均衡，但当其中一个Broker突然宕掉的话，那么存在于该Broker上处于Pending状态的message将会丢失，无法达到高可用的目的。
由于目前ActiveMQ官网上并没有一个明确的将两种部署方式相结合的部署方案，所以我采用了两种方式结合起来部署。

在本机完成2个broker的共享文件测试
2个broker activemq_node1和activemq_node2共享data文件夹。

[root@vanelife-mq1 data]# ll

total 47092
drwxr-xr-x. 11 root root 4096 Jul 11 2014 activemq_node1
drwxr-xr-x. 11 root root 4096 May 21 07:14 activemq_node2
drwxr-xr-x. 2 root root 4096 May 21 07:13 data

配置如下：

node1全配置

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
<!–
Licensed to the Apache Software Foundation (ASF) under one or more
contributor license agreements. See the NOTICE file distributed with
this work for additional information regarding copyright ownership.
The ASF licenses this file to You under the Apache License, Version 2.0
(the “License”); you may not use this file except in compliance with
the License. You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an “AS IS” BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
–>
<!– START SNIPPET: example –>
<beans
xmlns=”http://www.springframework.org/schema/beans”
xmlns:xsi=”http://www.w3.org/2001/XMLSchema-instance”
xsi:schemaLocation=”http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans.xsd
http://activemq.apache.org/schema/core http://activemq.apache.org/schema/core/activemq-core.xsd”>
<!– Allows us to use system properties as variables in this configuration file –>
<bean class=”org.springframework.beans.factory.config.PropertyPlaceholderConfigurer”>
<property name=”locations”>
<value>file:${activemq.conf}/credentials.properties</value>
</property>
</bean>
<!– Allows accessing the server log –>
<bean id=”logQuery” class=”org.fusesource.insight.log.log4j.Log4jLogQuery”
lazy-init=”false” scope=”singleton”
init-method=”start” destroy-method=”stop”>
</bean>
<!–
The <broker> element is used to configure the ActiveMQ broker.
–>
<broker xmlns=”http://activemq.apache.org/schema/core” brokerName=”broker1” dataDirectory=”${activemq.data}/data”>
<destinationPolicy>
<policyMap>
<policyEntries>
<policyEntry topic=”>” >
<!– The constantPendingMessageLimitStrategy is used to prevent
slow topic consumers to block producers and affect other consumers
by limiting the number of messages that are retained
For more information, see:
http://activemq.apache.org/slow-consumer-handling.html
–>
<pendingMessageLimitStrategy>
<constantPendingMessageLimitStrategy limit=”1000″/>
</pendingMessageLimitStrategy>
</policyEntry>
</policyEntries>
</policyMap>
</destinationPolicy>
<!–
The managementContext is used to configure how ActiveMQ is exposed in
JMX. By default, ActiveMQ uses the MBean server that is started by
the JVM. For more information, see:
http://activemq.apache.org/jmx.html
–>
<managementContext>
<managementContext createConnector=”true” connectorPort=”1100“/>
</managementContext>
<networkConnectors> 
<networkConnector uri=”multicast://default” 
dynamicOnly=”true” 
networkTTL=”3″ 
prefetchSize=”1″ 
decreaseNetworkConsumerPriority=”true” /> 
</networkConnectors>
<!–
Configure message persistence for the broker. The default persistence
mechanism is the KahaDB store (identified by the kahaDB tag).
For more information, see:
http://activemq.apache.org/persistence.html
–>
<persistenceAdapter>
<kahaDB directory=”/data/data“/>
</persistenceAdapter>
<!–
The systemUsage controls the maximum amount of space the broker will
use before disabling caching and/or slowing down producers. For more information, see:
http://activemq.apache.org/producer-flow-control.html
–>
<systemUsage>
<systemUsage>
<memoryUsage>
<memoryUsage percentOfJvmHeap=”70″ />
</memoryUsage>
<storeUsage>
<storeUsage limit=”100 gb”/>
</storeUsage>
<tempUsage>
<tempUsage limit=”50 gb”/>
</tempUsage>
</systemUsage>
</systemUsage>
<!–
The transport connectors expose ActiveMQ over a given protocol to
clients and other brokers. For more information, see:
http://activemq.apache.org/configuring-transports.html
–>
<transportConnectors>
<!– DOS protection, limit concurrent connections to 1000 and frame size to 100MB –>
<transportConnector name=”openwire” discoveryUri=”multicast://default” uri=”tcp://0.0.0.0:61616?maximumConnections=1000&amp;wireFormat.maxFrameSize=104857600″/>
<transportConnector name=”mqtt” uri=”mqtt://0.0.0.0:1883?maximumConnections=1000&amp;wireFormat.maxFrameSize=104857600″/>
</transportConnectors>
<!– destroy the spring context on shutdown to stop jetty –>
<shutdownHooks>
<bean xmlns=”http://www.springframework.org/schema/beans” class=”org.apache.activemq.hooks.SpringContextHook” />
</shutdownHooks>
</broker>
<!–
Enable web consoles, REST and Ajax APIs and demos
The web consoles requires by default login, you can disable this in the jetty.xml file
Take a look at ${ACTIVEMQ_HOME}/conf/jetty.xml for more details
–>
<import resource=”jetty.xml”/>
</beans>
<!– END SNIPPET: example –>

node2全配置

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
<!–
Licensed to the Apache Software Foundation (ASF) under one or more
contributor license agreements. See the NOTICE file distributed with
this work for additional information regarding copyright ownership.
The ASF licenses this file to You under the Apache License, Version 2.0
(the “License”); you may not use this file except in compliance with
the License. You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an “AS IS” BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
–>
<!– START SNIPPET: example –>
<beans
xmlns=”http://www.springframework.org/schema/beans”
xmlns:xsi=”http://www.w3.org/2001/XMLSchema-instance”
xsi:schemaLocation=”http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans.xsd
http://activemq.apache.org/schema/core http://activemq.apache.org/schema/core/activemq-core.xsd”>
<!– Allows us to use system properties as variables in this configuration file –>
<bean class=”org.springframework.beans.factory.config.PropertyPlaceholderConfigurer”>
<property name=”locations”>
<value>file:${activemq.conf}/credentials.properties</value>
</property>
</bean>
<!– Allows accessing the server log –>
<bean id=”logQuery” class=”org.fusesource.insight.log.log4j.Log4jLogQuery”
lazy-init=”false” scope=”singleton”
init-method=”start” destroy-method=”stop”>
</bean>
<!–
The <broker> element is used to configure the ActiveMQ broker.
–>
<broker xmlns=”http://activemq.apache.org/schema/core” brokerName=”broker2” dataDirectory=”${activemq.data}/data”>
<destinationPolicy>
<policyMap>
<policyEntries>
<policyEntry topic=”>” >
<!– The constantPendingMessageLimitStrategy is used to prevent
slow topic consumers to block producers and affect other consumers
by limiting the number of messages that are retained
For more information, see:
http://activemq.apache.org/slow-consumer-handling.html
–>
<pendingMessageLimitStrategy>
<constantPendingMessageLimitStrategy limit=”1000″/>
</pendingMessageLimitStrategy>
</policyEntry>
</policyEntries>
</policyMap>
</destinationPolicy>
<!–
The managementContext is used to configure how ActiveMQ is exposed in
JMX. By default, ActiveMQ uses the MBean server that is started by
the JVM. For more information, see:
http://activemq.apache.org/jmx.html
–>
<managementContext>
<managementContext createConnector=”true” connectorPort=”1101“/>
</managementContext>
<networkConnectors> 
<networkConnector uri=”multicast://default” 
dynamicOnly=”true” 
networkTTL=”3″ 
prefetchSize=”1″ 
decreaseNetworkConsumerPriority=”true” /> 
</networkConnectors>
<!–
Configure message persistence for the broker. The default persistence
mechanism is the KahaDB store (identified by the kahaDB tag).
For more information, see:
http://activemq.apache.org/persistence.html
–>
<persistenceAdapter>
<kahaDB directory=”/data/data“/>
</persistenceAdapter>
<!–
The systemUsage controls the maximum amount of space the broker will
use before disabling caching and/or slowing down producers. For more information, see:
http://activemq.apache.org/producer-flow-control.html
–>
<systemUsage>
<systemUsage>
<memoryUsage>
<memoryUsage percentOfJvmHeap=”70″ />
</memoryUsage>
<storeUsage>
<storeUsage limit=”100 gb”/>
</storeUsage>
<tempUsage>
<tempUsage limit=”50 gb”/>
</tempUsage>
</systemUsage>
</systemUsage>
<!–
The transport connectors expose ActiveMQ over a given protocol to
clients and other brokers. For more information, see:
http://activemq.apache.org/configuring-transports.html
–>
<transportConnectors>
<!– DOS protection, limit concurrent connections to 1000 and frame size to 100MB –>
<transportConnector name=”openwire” discoveryUri=”multicast://default” uri=”tcp://0.0.0.0:61617?maximumConnections=1000&amp;wireFormat.maxFrameSize=104857600″/>
<transportConnector name=”mqtt” uri=”mqtt://0.0.0.0:1884?maximumConnections=1000&amp;wireFormat.maxFrameSize=104857600″/>
</transportConnectors>
<!– destroy the spring context on shutdown to stop jetty –>
<shutdownHooks>
<bean xmlns=”http://www.springframework.org/schema/beans” class=”org.apache.activemq.hooks.SpringContextHook” />
</shutdownHooks>
</broker>
<!–
Enable web consoles, REST and Ajax APIs and demos
The web consoles requires by default login, you can disable this in the jetty.xml file
Take a look at ${ACTIVEMQ_HOME}/conf/jetty.xml for more details
–>
<import resource=”jetty.xml”/>
</beans>

记得修改jetty.xml的端口，两个broker的端口不要重复。

启动后，这样主从就做好了，平时这两个节点只有master在工作，slave待定状态，当master挂了之后，slave接管正常工作。

slave的日志正常没工作的情况下显示：

2015-0-21 09:04:29,637 | INFO | Database /data/data/lock is locked… waiting 10 seconds for the database to be unlocked. Reason: java.io.IOException: File ‘/data/data/lock’ could not be locked. | org.apache.activemq.store.SharedFileLocker | main
2015-05-21 09:04:39,640 | INFO | Database /data/data/lock is locked… waiting 10 seconds for the database to be unlocked. Reason: java.io.IOException: File ‘/data/data/lock’ could not be locked. |

做好了之后，再通过负载均衡复制多个这样的节点，高可用的集群就搭建完成了。