The Measurement of Integrated Human Service Network (The Children's Treatment Network of Simcoe York)

Abstract

Community-based human services have traditionally been provided by autonomous service agencies. They have their own funding source and independent process. Integration has been advocated as a key strategy to integrate different agencies together to provide multiple services for a targeted community. The Children's Treatment Network (CTN) of Simcoe York is a network of agencies and organizations providing services to children with multiple needs and their families in Simcoe County and York Region. This study was designed to evaluate the different levels of integrated service approaches for children on outcomes. The study consisted of two parts: phase I and phase II measurement. Our project covered phase I measurement with the following objectives. Clinically, we aimed to evaluate agencies' integration in the network, promote discussion, and determine any interrelationship between a network's integration and its functioning. The statistical objectives were to quantify the network integration for agency, to represent the overall integration, to quantify the association between network's integration and functioning and to assess the sensitivity of results. We measured agencies' integration through measuring its agreement in collaboration with other agencies in the network. The higher agreement in collaboration indicates a better services integration. We defined four different agreement measures from different perspectives. The agreement based on group's perception was defined to be the primary measure. We used mean difference, percentage and the Kappa statistic to measure the agreement for each agency. Correlation and regression analyses were applied in investigating the association between network's integration and its functioning. The sensitivity of the results was analyzed by examining the re-sampling bias of bootstrapping regression models. Agreement measures were consistent for each agency. In Simcoe, agencies had an average agreement 0.874 (S.D. 0.213) in mean difference, 46.63 (S.D. 12.91) in percentage and 0.258 (S.D. 0.143) in Kappa. Agencies of York had average agreements of 0.662 (S.D. 0.204), 49.36 (S.D. 13.06) and 0.282 (S.D. 0.121), respectively. Agency 10 and 33 in Simcoe and Agency 14 in York were found to have the highest agreement. Agency 3 and 21 in Simcoe and Agency 8 and 9 in York, on the other hand, were found to have the lowest agreement. Different graphical displays illustrated that the overall agreement in collaboration was low and the agencies in York generally had a higher agreement. Correlation analysis showed that synergy and agencies' perception of pros and cons were significantly correlated with the primary percentage agreement. In regression analysis, we did not find any significant functioning component. However, synergy was found to be much more associated with agreement than the other components. The estimates were 11.48% (-1.03%, 24.00%) and 11.21% (-2.48%, 24.90%) in un-weighted and weighted models respectively. Bootstrapping regression analysis showed that the results were robust to a change of sample. We concluded that the level of integration of CTN was low because the agencies generally had poor agreement in collaboration. Synergy was the most important component associated with the network's integration. Other functioning components detected were also associated with the integrating process but were less clinically important. We discussed the statistical approaches used in other contexts and some of their strength and weaknesses. We also considered some key limitations of the study. This study was a baseline measurement of CTN of Simcoe York for further analysis. The results provided a basis for future enhancement of integration of the network. Our experiences also provided ideas for improving design and analysis in integrated network measurement.